-- Logs begin at Wed 2022-11-02 06:45:43 UTC, end at Wed 2022-11-02 08:17:00 UTC. -- Nov 02 07:14:18.275303 dv-cinder-804757-31-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Nov 02 07:14:18.284494 dv-cinder-804757-31-iscsi devstack@keystone.service[81536]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Nov 02 07:14:18.286541 dv-cinder-804757-31-iscsi devstack@keystone.service[81536]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Nov 2 07:14:18 2022] *** Nov 02 07:14:18.286541 dv-cinder-804757-31-iscsi devstack@keystone.service[81536]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Nov 02 07:14:18.286541 dv-cinder-804757-31-iscsi devstack@keystone.service[81536]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Nov 02 07:14:18.286541 dv-cinder-804757-31-iscsi devstack@keystone.service[81536]: nodename: dv-cinder-804757-31-iscsi Nov 02 07:14:18.286541 dv-cinder-804757-31-iscsi devstack@keystone.service[81536]: machine: x86_64 Nov 02 07:14:18.286541 dv-cinder-804757-31-iscsi devstack@keystone.service[81536]: clock source: unix Nov 02 07:14:18.286541 dv-cinder-804757-31-iscsi devstack@keystone.service[81536]: pcre jit disabled Nov 02 07:14:18.286541 dv-cinder-804757-31-iscsi devstack@keystone.service[81536]: detected number of CPU cores: 4 Nov 02 07:14:18.286541 dv-cinder-804757-31-iscsi devstack@keystone.service[81536]: current working directory: / Nov 02 07:14:18.286541 dv-cinder-804757-31-iscsi devstack@keystone.service[81536]: detected binary path: /usr/bin/uwsgi-core Nov 02 07:14:18.287797 dv-cinder-804757-31-iscsi devstack@keystone.service[81536]: your processes number limit is 47680 Nov 02 07:14:18.287797 dv-cinder-804757-31-iscsi devstack@keystone.service[81536]: your memory page size is 4096 bytes Nov 02 07:14:18.287797 dv-cinder-804757-31-iscsi devstack@keystone.service[81536]: detected max file descriptor number: 2048 Nov 02 07:14:18.287797 dv-cinder-804757-31-iscsi devstack@keystone.service[81536]: lock engine: pthread robust mutexes Nov 02 07:14:18.287797 dv-cinder-804757-31-iscsi devstack@keystone.service[81536]: thunder lock: enabled Nov 02 07:14:18.287797 dv-cinder-804757-31-iscsi devstack@keystone.service[81536]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Nov 02 07:14:18.287797 dv-cinder-804757-31-iscsi devstack@keystone.service[81536]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Nov 02 07:14:18.328700 dv-cinder-804757-31-iscsi devstack@keystone.service[81536]: Python main interpreter initialized at 0x5596c76e59c0 Nov 02 07:14:18.328700 dv-cinder-804757-31-iscsi devstack@keystone.service[81536]: python threads support enabled Nov 02 07:14:18.328700 dv-cinder-804757-31-iscsi devstack@keystone.service[81536]: your server socket listen backlog is limited to 100 connections Nov 02 07:14:18.328700 dv-cinder-804757-31-iscsi devstack@keystone.service[81536]: your mercy for graceful operations on workers is 90 seconds Nov 02 07:14:18.328700 dv-cinder-804757-31-iscsi devstack@keystone.service[81536]: mapped 403077 bytes (393 KB) for 2 cores Nov 02 07:14:18.328700 dv-cinder-804757-31-iscsi devstack@keystone.service[81536]: *** Operational MODE: preforking *** Nov 02 07:14:18.328700 dv-cinder-804757-31-iscsi devstack@keystone.service[81536]: *** uWSGI is running in multiple interpreter mode *** Nov 02 07:14:18.328700 dv-cinder-804757-31-iscsi devstack@keystone.service[81536]: spawned uWSGI master process (pid: 81536) Nov 02 07:14:18.328700 dv-cinder-804757-31-iscsi devstack@keystone.service[81536]: spawned uWSGI worker 1 (pid: 81548, cores: 1) Nov 02 07:14:18.328700 dv-cinder-804757-31-iscsi devstack@keystone.service[81536]: spawned uWSGI worker 2 (pid: 81549, cores: 1) Nov 02 07:14:18.328700 dv-cinder-804757-31-iscsi devstack@keystone.service[81536]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Nov 02 07:14:18.327654 dv-cinder-804757-31-iscsi systemd[1]: Started Devstack devstack@keystone.service. Nov 02 07:14:22.559065 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Nov 02 07:14:22.559065 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: from cryptography.utils import int_from_bytes Nov 02 07:14:22.559896 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Nov 02 07:14:22.559896 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: from cryptography.utils import int_from_bytes Nov 02 07:14:22.559896 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Nov 02 07:14:22.559896 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: from cryptography.utils import int_from_bytes Nov 02 07:14:22.560602 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Nov 02 07:14:22.560602 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: from cryptography.utils import int_from_bytes Nov 02 07:14:22.609216 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Nov 02 07:14:22.609373 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Nov 02 07:14:22.609373 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Nov 02 07:14:22.609531 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Nov 02 07:14:22.609593 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] command line args: [] {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Nov 02 07:14:22.609653 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] command line args: [] {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Nov 02 07:14:22.609830 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Nov 02 07:14:22.609921 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Nov 02 07:14:22.609986 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ================================================================================ {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Nov 02 07:14:22.610048 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ================================================================================ {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Nov 02 07:14:22.610269 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] admin_token = **** {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.610407 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] admin_token = **** {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.610489 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] config_dir = [] {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.610555 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] config_dir = [] {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.610683 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.610824 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.610890 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] config_source = [] {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.610958 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] config_source = [] {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.611021 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] debug = True {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.611085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] debug = True {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.611235 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.612407 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.612407 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.612561 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.612619 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.612672 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.612726 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.612788 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.612844 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.612904 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.612967 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] list_limit = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.613025 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] list_limit = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.613085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] log_config_append = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.613151 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] log_config_append = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.613211 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.613271 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.613332 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] log_dir = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.613392 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] log_dir = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.613392 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] log_file = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.613504 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] log_file = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.613561 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.613620 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.613676 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.613735 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.613796 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.613859 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.614039 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.614114 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.614171 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.614233 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.614305 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.614369 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.614586 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.614730 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.614824 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.614908 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.615101 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.615216 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.615300 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.615370 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.615528 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.615666 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.615751 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.615814 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.615968 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.616056 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.616210 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.616404 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.616498 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.616577 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.616959 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.30/identity {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.617040 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.30/identity {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.617117 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] publish_errors = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.617187 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] publish_errors = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.617364 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.617487 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.617572 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.617646 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.617757 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.617835 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.617984 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.618126 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.618217 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.618287 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.618427 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] standard_threads = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.618514 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] standard_threads = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.618679 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.618761 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.618932 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.619014 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.619087 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.619165 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.619306 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] use_journal = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.619387 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] use_journal = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.619508 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] use_json = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.619618 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] use_json = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.619618 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] use_stderr = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.619742 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] use_stderr = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.619817 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] use_syslog = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.619882 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] use_syslog = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.619949 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.620016 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 07:14:22.620216 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.620298 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.620448 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.620527 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.620708 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.620786 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.620943 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.621020 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.621190 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.621269 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.621424 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.621502 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.621671 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.621750 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.621910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.621994 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.622152 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.622230 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.622396 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.622474 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.622646 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.622753 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.622967 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.623121 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.623220 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.623321 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.623459 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.623559 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.623710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.623808 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.623959 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.624056 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.624197 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.624280 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.624455 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.624536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.624709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.624785 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.624956 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.625040 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.625195 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.625271 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.625449 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.625537 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.625712 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.625788 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.625965 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.626045 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.626213 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.626299 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.626443 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.626524 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.626687 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.626765 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.626922 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.627005 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.627156 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.627230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.627403 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.627482 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.627651 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.627736 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.627899 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.627981 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.628104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.628179 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.628348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.628430 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.628594 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.628669 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.628828 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.628902 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.629074 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.629157 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.629301 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.629376 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.629535 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.629617 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.629770 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.629857 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.629996 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.630070 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.630226 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.630300 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.630451 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.630605 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.630690 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.630760 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.630924 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.631001 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.631170 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.631253 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.631401 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.631482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.631635 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.631710 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.631863 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.631944 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.632096 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.632201 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.632329 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.632433 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.632574 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.632692 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.632793 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.632884 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.633026 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.633126 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.633263 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.633389 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.633497 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.633576 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.633726 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.633807 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.633957 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.634072 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.634185 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.634260 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.634433 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.634511 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.634712 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.634794 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.634918 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.634992 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.635138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.635227 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.635394 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.635485 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.635633 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.635721 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.635865 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.635940 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.636099 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.636174 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.636334 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.636452 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.636600 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.636681 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.636835 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.636913 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.637080 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.637157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.637323 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.637404 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.637559 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.637635 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.637791 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.637870 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.638023 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.638099 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.638273 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.638348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.638556 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.638633 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.638822 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.638900 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.639068 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.639183 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.639309 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.639430 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.639562 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.639685 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.639800 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.639924 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.640035 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.640161 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.640271 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.640399 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.640507 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.640635 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.640742 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.640871 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.641003 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.641132 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.641245 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.641374 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.641493 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.641619 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.641735 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.641860 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.641972 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.642097 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.642249 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.642322 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.642439 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.642561 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.642713 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.642806 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.642931 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.643045 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.643164 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.643275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.643397 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.643504 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.643627 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.643733 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.643861 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.643966 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.644091 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.644194 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.644332 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.644436 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.644568 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.644670 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.644814 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.644916 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.645051 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.645138 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.645304 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.645380 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.645563 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.645639 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.645806 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.646270 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.646395 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] auth.external = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.646524 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] auth.external = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.646626 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.646717 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.646943 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.647031 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.647175 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.647261 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.647406 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] auth.password = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.647482 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] auth.password = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.647633 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] auth.token = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.647708 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] auth.token = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.647868 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.647951 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.648101 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.648175 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.648341 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.648419 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.648570 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.648654 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.648815 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.648891 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.649058 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.649135 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.649287 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.649374 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.649520 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] credential.caching = True {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.649596 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] credential.caching = True {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.649764 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.649844 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.650008 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.650092 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.650252 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.650327 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.650503 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.650579 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.650766 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.650856 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.650999 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.651075 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.651241 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.651318 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.651491 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.651575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.651726 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.651802 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.651975 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.652055 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.652258 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.652343 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.652500 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.652576 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.652756 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.652838 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.652997 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.653082 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.653251 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.653328 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.653483 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] federation.caching = True {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.653558 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] federation.caching = True {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.653717 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.653801 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.653959 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.654034 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.654231 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.654307 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.654464 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.654613 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.654737 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.654976 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.655052 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.655233 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.655320 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.655471 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.655547 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.655715 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.655797 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.655951 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.656035 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.656194 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.656269 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.656429 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] identity.caching = True {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.656506 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.656664 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] identity.caching = True {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.656821 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.656915 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.656991 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.657152 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.657229 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.657384 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.657468 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.657624 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.657701 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.657852 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.657928 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.658087 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.658171 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.658324 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.658402 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.658571 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.658647 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.658828 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.658916 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.659061 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.659138 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.659289 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.659364 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.659526 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.659610 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.659769 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.659844 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.660012 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.660087 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.660259 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.660578 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.660578 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.660725 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.660725 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.660851 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.661000 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.661104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.661219 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.661300 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.661449 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.661525 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.661678 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.661762 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.661906 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.661982 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.662134 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.662209 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.662370 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.662453 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.662607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.662730 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.662871 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.662958 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.663103 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.663187 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.663335 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.663417 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.663581 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.663658 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.663816 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.663899 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.664049 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.664124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.664294 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.664370 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.664522 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.664599 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.664756 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.664832 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.664983 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.665059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.665208 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.665285 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.665445 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.665521 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.665682 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.665795 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.666032 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.666114 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.666270 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.666345 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.666507 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.666583 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.666784 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.666864 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.667021 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.667107 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.667253 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.667582 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.667582 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.667723 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.667793 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.667863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.667953 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.668030 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.668188 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.668265 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.668422 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.668498 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.668651 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.user = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.668726 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.668880 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.user = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.668957 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.669120 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.669198 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.669352 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.669428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.669604 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.669681 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.669907 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.670012 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.670153 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.670279 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.670377 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.670450 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.670603 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.670736 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.670860 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.670936 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.671095 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.671174 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.671326 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.671402 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.671553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.671629 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.671779 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.671857 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.672025 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.672101 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.672267 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.672342 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.672508 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.672585 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.672751 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.672826 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.672981 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.673058 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.673210 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.673287 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.673450 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.673527 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.673683 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.673760 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.673918 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.673995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.674150 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.674226 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.674385 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.674468 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.674629 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.674770 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.674889 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.674966 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.675123 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.675199 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.675365 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.675443 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.675593 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.675676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.675831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.675913 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.676062 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.676139 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.676296 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.676374 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.676531 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.676615 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.676770 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.676850 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.677002 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.677079 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.677230 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.677306 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.677463 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.677542 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] resource.caching = True {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.677701 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] resource.caching = True {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.677778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.677944 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.678019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.678180 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.678258 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.678410 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.678487 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.678701 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.678782 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.678913 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.678992 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.679150 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.679226 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.679382 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.679458 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.679605 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.679678 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.679844 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.679921 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] role.caching = True {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.680075 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] role.caching = True {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.680160 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] role.driver = sql {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.680303 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.680379 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] role.driver = sql {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.680534 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.680610 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.680783 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.680867 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.681026 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.681104 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.681271 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.681347 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.681513 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.681597 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.681760 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.681837 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.682005 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.682081 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.682248 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.682332 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.682482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.682558 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.682728 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.682808 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.682977 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.683065 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.683222 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.683300 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.683465 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.683541 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.683707 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.683860 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.684098 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.684174 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.684334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.684410 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.684583 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.684667 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.684831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.684907 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.685078 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.685153 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.685310 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.685438 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.685589 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.685670 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.685844 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.686006 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.686087 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.686159 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.686305 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.686383 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.686537 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.686619 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.686790 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.686870 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.687020 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.687097 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.687269 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.687346 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.687513 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.687595 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.687751 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.687828 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.687984 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.688060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.688213 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.688291 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.688441 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.688518 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] token.caching = True {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.688675 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] token.caching = True {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.688751 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.688913 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.688992 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.689145 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.689220 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.689383 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.689461 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.689637 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.689714 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.689873 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.689950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.690104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.690179 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.690344 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.690421 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.690583 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.690675 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.690841 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.690916 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.691075 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.691153 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.691307 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.691382 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.691537 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.691613 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.691781 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.691859 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.692026 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.692110 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.692268 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.692357 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.692508 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.692587 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.692769 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] database.connection = **** {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.692853 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.693014 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.693091 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] database.connection = **** {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.693254 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.693330 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.693487 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.693577 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.693722 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.693800 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.693955 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.694034 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.694187 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.694271 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.694417 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.694493 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.694649 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.694744 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.694914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.695003 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.695150 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.695226 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.695387 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.695463 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.695621 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.695705 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.695855 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.695931 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.696090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.696166 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.696323 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.696406 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.696553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.696628 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.696782 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.696859 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.697015 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.697099 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.697170 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Nov 02 07:14:22.697239 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 07:14:22.697402 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Nov 02 07:14:22.726517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:22.726692 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:22.726763 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:22.726822 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:22.726986 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:22.727068 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:22.735604 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:22.735696 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:22.764798 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:22.765001 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:22.765081 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:22.765309 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:22.816802 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:22.816959 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:22.864534 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:22.864790 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:22.865370 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:22.865609 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.023317 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.023523 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.023589 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.023751 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.023901 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.023972 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.024041 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.024144 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.024230 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.024375 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.024508 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.024579 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.047503 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.047635 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.047705 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.047835 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.047928 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.048058 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.048149 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.048286 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.048387 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.048493 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.048583 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.048738 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.048868 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.048953 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.059813 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.060081 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.072990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.073126 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.073196 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.073357 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.073451 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.073555 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.073644 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.073789 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.073931 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.074002 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.074085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.074159 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.074220 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.074374 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 07:14:23.091055 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.091258 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Nov 02 07:14:23.091382 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.091596 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.091795 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Nov 02 07:14:23.091900 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Nov 02 07:14:23.091969 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.092183 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.092405 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Nov 02 07:14:23.092478 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.092568 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.092691 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.092866 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.093134 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.093258 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.093453 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.102323 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.102604 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.102753 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.102871 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.103033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.103179 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.103296 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.103612 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.110145 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.110550 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.113005 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.113288 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.113413 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.113568 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.113694 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.113845 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.113986 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.114102 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.114194 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.114387 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.114518 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.114633 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.114760 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.114960 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.115091 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.115317 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.115461 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.115610 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.115734 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.115857 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.115945 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.116218 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.138042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.138529 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.139534 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.139818 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.140161 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.140448 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.146411 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.146692 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.147002 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.147160 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.147486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.147775 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.153479 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.153762 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.154486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.154874 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.157894 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.158183 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.158465 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.158781 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.158989 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.159105 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.159314 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.159440 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.159574 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.159696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.159872 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.159997 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.160146 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.160440 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.160721 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.160999 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.175147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.175402 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.176479 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.176763 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.179228 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.180610 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.182380 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.182689 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.183825 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.184123 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.186913 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.188438 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.190008 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.191627 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.193210 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.193508 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.193808 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.194100 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.194384 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.194689 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.194831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.195116 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.195420 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.195719 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.196014 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.196279 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.210179 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.210465 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.210758 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.211020 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.212049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.212340 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.212618 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.212853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.217795 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.219340 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.219869 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.220871 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.221147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.221407 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.223075 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.223318 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.223812 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.224096 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.224500 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.224793 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.225075 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.225938 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.226250 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.226625 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.226942 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.227237 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.235969 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.236256 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.236627 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.236911 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.237207 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.237477 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.237755 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.238235 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.238537 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.238938 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.239254 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.239528 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.239818 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.240096 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.252904 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.255419 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.255753 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.258286 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.258598 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.260040 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.261196 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.261630 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.261936 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.262207 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.262619 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.264271 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.264562 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.264849 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.268429 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.269889 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.271490 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.272968 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.273162 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.273473 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.273758 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.274043 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.275959 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.276253 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.276538 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.276828 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.281761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.282049 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.282332 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.284616 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.284917 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.285203 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.289948 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.290235 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.290518 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.290816 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.291087 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.291352 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.291632 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.291945 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.292235 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.292478 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.292754 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.292930 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 07:14:23.293073 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.293205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.293331 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.293498 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.293632 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.293749 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.294037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.294320 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.294606 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.294985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.295276 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.295551 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.295812 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.296087 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.296366 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.296631 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 07:14:23.326495 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5596c76e59c0 pid: 81549 (default app) Nov 02 07:14:23.329251 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5596c76e59c0 pid: 81548 (default app) Nov 02 07:14:23.329722 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc47452-952a-4577-9b0f-bfd7bb70adb8 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:23.330006 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc47452-952a-4577-9b0f-bfd7bb70adb8 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:23.330273 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc47452-952a-4577-9b0f-bfd7bb70adb8 None None] PATH_INFO: `/v3` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:23.334416 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1/1] 10.222.0.30 () {50 vars in 791 bytes} [Wed Nov 2 07:14:23 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:30.128847 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8844e-f718-4ca5-918d-e304532a6345 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:30.129394 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8844e-f718-4ca5-918d-e304532a6345 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:30.129584 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8844e-f718-4ca5-918d-e304532a6345 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:30.136314 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1/2] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:14:30 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:14:30.143726 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203380a-d4a6-40e0-ab49-6cc86a2d38b7 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:30.144255 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203380a-d4a6-40e0-ab49-6cc86a2d38b7 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:30.144255 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203380a-d4a6-40e0-ab49-6cc86a2d38b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:30.227220 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: INFO dbcounter [None req-6203380a-d4a6-40e0-ab49-6cc86a2d38b7 None None] Registered counter for database keystone Nov 02 07:14:30.249554 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG oslo_db.sqlalchemy.engines [None req-6203380a-d4a6-40e0-ab49-6cc86a2d38b7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81549) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Nov 02 07:14:30.258740 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writer thread running {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Nov 02 07:14:30.448179 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG passlib.handlers.bcrypt [None req-6203380a-d4a6-40e0-ab49-6cc86a2d38b7 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81549) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Nov 02 07:14:30.448499 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG passlib.handlers.bcrypt [None req-6203380a-d4a6-40e0-ab49-6cc86a2d38b7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81549) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Nov 02 07:14:30.511793 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-6203380a-d4a6-40e0-ab49-6cc86a2d38b7 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:30.513478 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-6203380a-d4a6-40e0-ab49-6cc86a2d38b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:30.662416 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 2/3] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:30 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 520 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Nov 02 07:14:31.138637 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca8844e-f718-4ca5-918d-e304532a6345 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:31.221400 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: INFO dbcounter [None req-9ca8844e-f718-4ca5-918d-e304532a6345 None None] Registered counter for database keystone Nov 02 07:14:31.246795 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG oslo_db.sqlalchemy.engines [None req-9ca8844e-f718-4ca5-918d-e304532a6345 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81548) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Nov 02 07:14:31.256648 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writer thread running {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Nov 02 07:14:31.374038 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca8844e-f718-4ca5-918d-e304532a6345 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:31.379436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe7eb9-5910-49ad-b93f-aeb1dab1e4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:31.380051 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe7eb9-5910-49ad-b93f-aeb1dab1e4ac admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:31.380322 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe7eb9-5910-49ad-b93f-aeb1dab1e4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:31.380590 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe7eb9-5910-49ad-b93f-aeb1dab1e4ac admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:31.389067 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe7eb9-5910-49ad-b93f-aeb1dab1e4ac admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:31.439860 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e0fe7eb9-5910-49ad-b93f-aeb1dab1e4ac admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Nov 02 07:14:31.440204 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe7eb9-5910-49ad-b93f-aeb1dab1e4ac admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:31.454175 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 2/4] 10.222.0.30 () {58 vars in 1324 bytes} [Wed Nov 2 07:14:31 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:14:31.463154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203380a-d4a6-40e0-ab49-6cc86a2d38b7 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:31.481988 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203380a-d4a6-40e0-ab49-6cc86a2d38b7 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:31.487865 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e425b8-383a-4f8f-a617-ca8321c70706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:31.488987 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e425b8-383a-4f8f-a617-ca8321c70706 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:31.489268 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e425b8-383a-4f8f-a617-ca8321c70706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:31.489538 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e425b8-383a-4f8f-a617-ca8321c70706 admin admin] PATH_INFO: `/v3/projects/b3f647c33aef4316a529d7730281cb6b` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:31.493509 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e425b8-383a-4f8f-a617-ca8321c70706 admin admin] RBAC: Authorizing `identity:get_project(project_id=b3f647c33aef4316a529d7730281cb6b)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:31.543563 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: WARNING keystone.common.rbac_enforcer.enforcer [None req-17e425b8-383a-4f8f-a617-ca8321c70706 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Nov 02 07:14:31.543906 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e425b8-383a-4f8f-a617-ca8321c70706 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:31.548418 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 3/5] 10.222.0.30 () {56 vars in 1207 bytes} [Wed Nov 2 07:14:31 2022] GET /identity/v3/projects/b3f647c33aef4316a529d7730281cb6b => generated 344 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:33.680832 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1abc6-263e-4df0-9612-28b428d912cb None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:33.681368 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1abc6-263e-4df0-9612-28b428d912cb None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:33.681368 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1abc6-263e-4df0-9612-28b428d912cb None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:33.685498 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 3/6] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:14:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:33.694001 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9019f14-25d4-4eb4-b12b-a18056e6cff1 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:33.694533 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9019f14-25d4-4eb4-b12b-a18056e6cff1 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:33.694533 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9019f14-25d4-4eb4-b12b-a18056e6cff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:33.724315 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-b9019f14-25d4-4eb4-b12b-a18056e6cff1 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:33.725851 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-b9019f14-25d4-4eb4-b12b-a18056e6cff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:33.734976 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 4/7] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:33 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Nov 02 07:14:34.004128 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e1abc6-263e-4df0-9612-28b428d912cb None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:34.018583 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e1abc6-263e-4df0-9612-28b428d912cb None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:34.024139 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1478f3b-3d0c-4808-96b2-53523038d18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:34.024737 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1478f3b-3d0c-4808-96b2-53523038d18b admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:34.025021 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1478f3b-3d0c-4808-96b2-53523038d18b admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:34.025294 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1478f3b-3d0c-4808-96b2-53523038d18b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:34.033810 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1478f3b-3d0c-4808-96b2-53523038d18b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:34.035630 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1478f3b-3d0c-4808-96b2-53523038d18b admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:34.045109 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 4/8] 10.222.0.30 () {58 vars in 1324 bytes} [Wed Nov 2 07:14:34 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Nov 02 07:14:34.051440 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9019f14-25d4-4eb4-b12b-a18056e6cff1 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:34.066419 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9019f14-25d4-4eb4-b12b-a18056e6cff1 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:34.075291 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf550c-b5d6-4900-a663-2a9a17085906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:34.076219 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf550c-b5d6-4900-a663-2a9a17085906 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:34.076599 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf550c-b5d6-4900-a663-2a9a17085906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:34.076949 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf550c-b5d6-4900-a663-2a9a17085906 admin admin] PATH_INFO: `/v3/users/8715d21c18b14387953f25f094b18c6d` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:34.079239 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf550c-b5d6-4900-a663-2a9a17085906 admin admin] RBAC: Authorizing `identity:get_user(user_id=8715d21c18b14387953f25f094b18c6d)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:34.081542 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf550c-b5d6-4900-a663-2a9a17085906 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:34.083122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 5/9] 10.222.0.30 () {56 vars in 1198 bytes} [Wed Nov 2 07:14:34 2022] GET /identity/v3/users/8715d21c18b14387953f25f094b18c6d => generated 247 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:36.289632 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c3173-0d89-4eab-9843-6fcdbe749f2a None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:36.290209 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c3173-0d89-4eab-9843-6fcdbe749f2a None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:36.290209 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c3173-0d89-4eab-9843-6fcdbe749f2a None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:36.291542 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 5/10] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:36.296927 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6d13b-2d53-4d6d-9fff-e347d005ad5a None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:36.297366 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6d13b-2d53-4d6d-9fff-e347d005ad5a None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:36.297454 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6d13b-2d53-4d6d-9fff-e347d005ad5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:36.308567 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc2b17-c149-4e12-8859-148851a53a2e None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:36.308860 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc2b17-c149-4e12-8859-148851a53a2e None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:36.309118 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc2b17-c149-4e12-8859-148851a53a2e None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:36.310389 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 6/11] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:36.316006 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e42fb-8f01-4ae9-84ae-fb2d6aa1d052 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:36.316508 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e42fb-8f01-4ae9-84ae-fb2d6aa1d052 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:36.316806 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e42fb-8f01-4ae9-84ae-fb2d6aa1d052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:36.317414 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-fcb6d13b-2d53-4d6d-9fff-e347d005ad5a None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:36.318615 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-fcb6d13b-2d53-4d6d-9fff-e347d005ad5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:36.360491 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG passlib.handlers.bcrypt [None req-390e42fb-8f01-4ae9-84ae-fb2d6aa1d052 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81548) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Nov 02 07:14:36.360881 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG passlib.handlers.bcrypt [None req-390e42fb-8f01-4ae9-84ae-fb2d6aa1d052 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81548) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Nov 02 07:14:36.403601 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 6/12] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:36.409679 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-390e42fb-8f01-4ae9-84ae-fb2d6aa1d052 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:36.411304 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-390e42fb-8f01-4ae9-84ae-fb2d6aa1d052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:36.513790 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 7/13] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:36.668339 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6d13b-2d53-4d6d-9fff-e347d005ad5a None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:36.694564 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:36.774489 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e42fb-8f01-4ae9-84ae-fb2d6aa1d052 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:36.776731 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6d13b-2d53-4d6d-9fff-e347d005ad5a None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:36.811473 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c045045-5f0c-4961-ba33-c454e8417f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:36.812116 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c045045-5f0c-4961-ba33-c454e8417f37 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:36.812396 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c045045-5f0c-4961-ba33-c454e8417f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:36.812670 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c045045-5f0c-4961-ba33-c454e8417f37 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:36.824092 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c045045-5f0c-4961-ba33-c454e8417f37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:36.832937 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-8c045045-5f0c-4961-ba33-c454e8417f37 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:14:36.832937 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:36.832937 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:36.832937 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:36.832937 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:36.832937 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:36.832937 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:36.832937 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:36.832937 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:36.832937 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:36.832937 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:36.832937 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:36.832937 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:36.832937 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:36.832937 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:36.832937 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:36.833888 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:36.833888 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:36.833888 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:36.833888 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:36.833888 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:36.833888 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:36.833888 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:36.833888 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:36.833888 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:36.833888 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:36.833888 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:36.833888 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:36.833888 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:36.833888 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:36.833888 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:36.833888 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:36.834749 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:36.834749 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:36.834749 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:36.834749 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:36.834749 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:36.834749 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:36.834749 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:36.834749 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:36.834749 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:36.834749 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:36.834749 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:14:36.834749 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:14:36.834749 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 7/14] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 07:14:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:36.840007 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c045045-5f0c-4961-ba33-c454e8417f37 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:36.876099 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e42fb-8f01-4ae9-84ae-fb2d6aa1d052 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:36.878501 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:36.913397 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835fb97c-70c9-4fd7-8a4b-0ec20d1ade24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:36.914270 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835fb97c-70c9-4fd7-8a4b-0ec20d1ade24 None admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:36.914550 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835fb97c-70c9-4fd7-8a4b-0ec20d1ade24 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:36.914857 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835fb97c-70c9-4fd7-8a4b-0ec20d1ade24 None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:36.915828 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835fb97c-70c9-4fd7-8a4b-0ec20d1ade24 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:36.917272 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835fb97c-70c9-4fd7-8a4b-0ec20d1ade24 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:36.926310 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:36.929459 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.sql.core [None req-835fb97c-70c9-4fd7-8a4b-0ec20d1ade24 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 07:14:36.929459 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 07:14:36.929459 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [parameters: {'id': '8d01c2e939d4423693823aa7344bd6a2', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Nov 02 07:14:36.929459 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81548) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Nov 02 07:14:36.945157 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c045045-5f0c-4961-ba33-c454e8417f37 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:36.948575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-835fb97c-70c9-4fd7-8a4b-0ec20d1ade24 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Nov 02 07:14:36.948575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:36.948575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:14:36.948575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:14:36.948575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:14:36.948575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:14:36.948575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:14:36.948575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:14:36.948575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:14:36.948575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:14:36.948575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:14:36.948575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:14:36.948575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:14:36.948575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application result.read() Nov 02 07:14:36.949339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:14:36.949339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:14:36.949339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:14:36.949339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:14:36.949339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:14:36.949339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:14:36.949339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:14:36.949339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:14:36.949339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 07:14:36.949339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:14:36.949339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Nov 02 07:14:36.949339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:14:36.949339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:36.949339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Nov 02 07:14:36.949339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return method(*args, **kwargs) Nov 02 07:14:36.949339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Nov 02 07:14:36.949339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return project_ref.to_dict() Nov 02 07:14:36.949339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:14:36.950205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:14:36.950205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Nov 02 07:14:36.950205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application yield resource Nov 02 07:14:36.950205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:14:36.950205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:14:36.950205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Nov 02 07:14:36.950205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self.session.rollback() Nov 02 07:14:36.950205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Nov 02 07:14:36.950205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self.force_reraise() Nov 02 07:14:36.950205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Nov 02 07:14:36.950205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise self.value Nov 02 07:14:36.950205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Nov 02 07:14:36.950205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Nov 02 07:14:36.950205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Nov 02 07:14:36.950205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application session.commit() Nov 02 07:14:36.950205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Nov 02 07:14:36.950205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Nov 02 07:14:36.951107 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Nov 02 07:14:36.951107 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._prepare_impl() Nov 02 07:14:36.951107 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Nov 02 07:14:36.951107 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self.session.flush() Nov 02 07:14:36.951107 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Nov 02 07:14:36.951107 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._flush(objects) Nov 02 07:14:36.951107 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Nov 02 07:14:36.951107 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Nov 02 07:14:36.951107 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Nov 02 07:14:36.951107 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application compat.raise_( Nov 02 07:14:36.951107 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:14:36.951107 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception Nov 02 07:14:36.951107 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Nov 02 07:14:36.951107 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application flush_context.execute() Nov 02 07:14:36.951107 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Nov 02 07:14:36.951107 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rec.execute(self) Nov 02 07:14:36.951107 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Nov 02 07:14:36.951964 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Nov 02 07:14:36.951964 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Nov 02 07:14:36.951964 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application _emit_insert_statements( Nov 02 07:14:36.951964 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Nov 02 07:14:36.951964 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application c = connection._execute_20( Nov 02 07:14:36.951964 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Nov 02 07:14:36.951964 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Nov 02 07:14:36.951964 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Nov 02 07:14:36.951964 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Nov 02 07:14:36.951964 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Nov 02 07:14:36.951964 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ret = self._execute_context( Nov 02 07:14:36.951964 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Nov 02 07:14:36.951964 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Nov 02 07:14:36.951964 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Nov 02 07:14:36.951964 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Nov 02 07:14:36.952733 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:14:36.952733 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception Nov 02 07:14:36.952733 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:14:36.952733 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:14:36.952733 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:14:36.952733 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:14:36.952733 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:14:36.952733 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:14:36.952733 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:14:36.952733 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:14:36.952733 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:14:36.952733 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:14:36.952733 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:14:36.952733 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application result.read() Nov 02 07:14:36.952733 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:14:36.952733 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:14:36.952733 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:14:36.953714 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:14:36.953714 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:14:36.953714 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:14:36.953714 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:14:36.953714 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:14:36.953714 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 07:14:36.953714 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 07:14:36.953714 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [parameters: {'id': '8d01c2e939d4423693823aa7344bd6a2', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Nov 02 07:14:36.953714 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Nov 02 07:14:36.953714 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:14:36.953714 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:14:36.953714 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:14:36.953714 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:36.953714 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Nov 02 07:14:36.953714 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Nov 02 07:14:36.955309 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Nov 02 07:14:36.955309 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Nov 02 07:14:36.955309 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Nov 02 07:14:36.955309 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:14:36.955309 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:14:36.955309 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:14:36.955309 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:36.955309 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:36.955309 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:36.955309 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:36.955309 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:36.955309 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:36.955309 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:36.955309 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:36.955309 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:36.955309 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:36.955309 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:36.956896 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Nov 02 07:14:36.956896 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Nov 02 07:14:36.956896 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:36.956896 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:36.956896 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Nov 02 07:14:36.956896 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Nov 02 07:14:36.956896 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:36.956896 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:36.956896 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Nov 02 07:14:36.956896 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.Conflict( Nov 02 07:14:36.956896 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Nov 02 07:14:36.956896 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:14:36.956896 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 8/15] 10.222.0.30 () {60 vars in 1137 bytes} [Wed Nov 2 07:14:36 2022] POST /identity/v3/domains => generated 190 bytes in 176 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:14:36.956896 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835fb97c-70c9-4fd7-8a4b-0ec20d1ade24 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:36.982132 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f640263-d084-434d-a9c9-23db603a9df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:36.982780 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f640263-d084-434d-a9c9-23db603a9df4 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:36.983065 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f640263-d084-434d-a9c9-23db603a9df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:36.983332 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f640263-d084-434d-a9c9-23db603a9df4 None admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:36.983998 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f640263-d084-434d-a9c9-23db603a9df4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:36.984297 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f640263-d084-434d-a9c9-23db603a9df4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:36.985728 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f640263-d084-434d-a9c9-23db603a9df4 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:36.997576 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 8/16] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:37.003397 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f640263-d084-434d-a9c9-23db603a9df4 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:37.057977 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835fb97c-70c9-4fd7-8a4b-0ec20d1ade24 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:37.093375 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9afcaa-a90d-481a-bd01-57228e58c627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:37.094234 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9afcaa-a90d-481a-bd01-57228e58c627 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:37.094505 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9afcaa-a90d-481a-bd01-57228e58c627 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:37.094797 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9afcaa-a90d-481a-bd01-57228e58c627 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:37.104306 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f640263-d084-434d-a9c9-23db603a9df4 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:37.113272 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9afcaa-a90d-481a-bd01-57228e58c627 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:37.114808 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9afcaa-a90d-481a-bd01-57228e58c627 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:37.131237 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-0e9afcaa-a90d-481a-bd01-57228e58c627 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:37.131237 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:37.131237 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:37.131237 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:37.131237 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:37.131237 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:37.131237 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:37.131237 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:37.131237 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:37.131237 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:37.131237 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:37.131237 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:37.131237 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:37.131237 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:37.131237 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:37.131237 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:37.132250 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:37.132250 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:37.132250 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:37.132250 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:37.132250 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:37.132250 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:37.132250 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:37.132250 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:37.132250 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:37.132250 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:37.132250 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:37.132250 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:37.132250 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:37.132250 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:37.132250 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:37.132250 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:37.132250 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:37.133325 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:37.133325 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:37.133325 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:14:37.133325 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 9/17] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:14:37.138446 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9afcaa-a90d-481a-bd01-57228e58c627 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:37.139136 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ec2e8-8bb4-4eb8-9e38-d009648546a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:37.139775 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ec2e8-8bb4-4eb8-9e38-d009648546a1 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:37.140053 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ec2e8-8bb4-4eb8-9e38-d009648546a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:37.140321 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ec2e8-8bb4-4eb8-9e38-d009648546a1 None admin] PATH_INFO: `/v3/users/8715d21c18b14387953f25f094b18c6d` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:37.141445 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ec2e8-8bb4-4eb8-9e38-d009648546a1 None admin] RBAC: Authorizing `identity:get_user(user_id=8715d21c18b14387953f25f094b18c6d)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:37.142935 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ec2e8-8bb4-4eb8-9e38-d009648546a1 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:37.144575 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 9/18] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:37 2022] GET /identity/v3/users/8715d21c18b14387953f25f094b18c6d => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:37.150476 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ec2e8-8bb4-4eb8-9e38-d009648546a1 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:37.165538 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:37.165791 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:37.247930 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9afcaa-a90d-481a-bd01-57228e58c627 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:37.249621 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ec2e8-8bb4-4eb8-9e38-d009648546a1 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:37.282728 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ab8c40-f7f1-403f-8303-70e5572924d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:37.283319 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab8c40-f7f1-403f-8303-70e5572924d0 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:37.283474 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6bec36-40c0-4378-8812-afb01d5afa29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:37.283582 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab8c40-f7f1-403f-8303-70e5572924d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:37.283861 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab8c40-f7f1-403f-8303-70e5572924d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:37.284352 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6bec36-40c0-4378-8812-afb01d5afa29 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:37.284697 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6bec36-40c0-4378-8812-afb01d5afa29 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:37.284792 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ab8c40-f7f1-403f-8303-70e5572924d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:37.284992 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ab8c40-f7f1-403f-8303-70e5572924d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:37.285079 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6bec36-40c0-4378-8812-afb01d5afa29 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:37.286282 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6bec36-40c0-4378-8812-afb01d5afa29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:37.286380 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ab8c40-f7f1-403f-8303-70e5572924d0 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:37.287934 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6bec36-40c0-4378-8812-afb01d5afa29 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:37.289498 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 10/19] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:37.295134 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6bec36-40c0-4378-8812-afb01d5afa29 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:37.303641 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 10/20] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:37.382927 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:37.405332 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6bec36-40c0-4378-8812-afb01d5afa29 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:37.442137 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63acce07-32d4-470b-993a-59ab6cb305f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:37.442860 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63acce07-32d4-470b-993a-59ab6cb305f7 None admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:37.443139 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63acce07-32d4-470b-993a-59ab6cb305f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:37.443409 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63acce07-32d4-470b-993a-59ab6cb305f7 None admin] PATH_INFO: `/v3/domains/default/users/8715d21c18b14387953f25f094b18c6d/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:37.445493 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63acce07-32d4-470b-993a-59ab6cb305f7 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=8715d21c18b14387953f25f094b18c6d, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:37.447032 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63acce07-32d4-470b-993a-59ab6cb305f7 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:37.563491 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 11/21] 10.222.0.30 () {58 vars in 1361 bytes} [Wed Nov 2 07:14:37 2022] PUT /identity/v3/domains/default/users/8715d21c18b14387953f25f094b18c6d/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:14:39.926184 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29acab88-7364-4010-a6ee-0640fedbae13 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:39.926694 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29acab88-7364-4010-a6ee-0640fedbae13 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:39.926694 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29acab88-7364-4010-a6ee-0640fedbae13 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:39.928116 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 11/22] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:39.937524 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074ff07-5a20-4847-8789-8bd7aa20556e None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:39.937948 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074ff07-5a20-4847-8789-8bd7aa20556e None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:39.938119 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074ff07-5a20-4847-8789-8bd7aa20556e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:39.963798 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-1074ff07-5a20-4847-8789-8bd7aa20556e None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:39.965328 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-1074ff07-5a20-4847-8789-8bd7aa20556e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:40.044288 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 12/23] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:14:40.312644 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b364f7d-b414-49fb-accd-78411247dafa None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:40.312983 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b364f7d-b414-49fb-accd-78411247dafa None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:40.313080 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b364f7d-b414-49fb-accd-78411247dafa None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:40.314411 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 12/24] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:40.320172 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbf93c-ec04-47b9-b28d-916957b10793 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:40.320457 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbf93c-ec04-47b9-b28d-916957b10793 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:40.320729 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbf93c-ec04-47b9-b28d-916957b10793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:40.343763 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7929e1-3b3c-4715-a89f-5110ca879fc8 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:40.344066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7929e1-3b3c-4715-a89f-5110ca879fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:40.344308 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7929e1-3b3c-4715-a89f-5110ca879fc8 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:40.345842 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 13/25] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:40.346307 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7929e1-3b3c-4715-a89f-5110ca879fc8 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:40.351732 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-27fbf93c-ec04-47b9-b28d-916957b10793 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:40.352919 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-27fbf93c-ec04-47b9-b28d-916957b10793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:40.440614 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:40.463663 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7929e1-3b3c-4715-a89f-5110ca879fc8 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:40.500114 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:40.506376 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 13/26] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:40.508975 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f1429c-001b-405c-abc9-a194c5bccd10 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:40.510377 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f1429c-001b-405c-abc9-a194c5bccd10 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:40.510877 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f1429c-001b-405c-abc9-a194c5bccd10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:40.516840 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee671e-677b-4ac7-860c-5f9bd2dd04c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:40.517738 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee671e-677b-4ac7-860c-5f9bd2dd04c4 None admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:40.518808 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee671e-677b-4ac7-860c-5f9bd2dd04c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:40.519621 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee671e-677b-4ac7-860c-5f9bd2dd04c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:40.521457 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fee671e-677b-4ac7-860c-5f9bd2dd04c4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:40.525584 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fee671e-677b-4ac7-860c-5f9bd2dd04c4 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:40.543784 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-11f1429c-001b-405c-abc9-a194c5bccd10 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:40.545664 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-11f1429c-001b-405c-abc9-a194c5bccd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:40.694834 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 14/27] 10.222.0.30 () {60 vars in 1131 bytes} [Wed Nov 2 07:14:40 2022] POST /identity/v3/roles => generated 219 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:14:40.701641 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 14/28] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:40.752216 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76fd68-79b6-4b82-b358-0e149516e4ea None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:40.752498 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76fd68-79b6-4b82-b358-0e149516e4ea None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:40.752929 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76fd68-79b6-4b82-b358-0e149516e4ea None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:40.754826 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 15/29] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:14:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:40.760916 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75ee75-874e-44ca-930c-20d4404adda2 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:40.761200 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75ee75-874e-44ca-930c-20d4404adda2 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:40.761628 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75ee75-874e-44ca-930c-20d4404adda2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:40.785174 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-6f75ee75-874e-44ca-930c-20d4404adda2 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:40.786688 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-6f75ee75-874e-44ca-930c-20d4404adda2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:40.880625 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 15/30] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:40.980360 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f76fd68-79b6-4b82-b358-0e149516e4ea None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:41.059731 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f75ee75-874e-44ca-930c-20d4404adda2 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:41.118341 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f76fd68-79b6-4b82-b358-0e149516e4ea None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:41.165120 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:41.183513 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb547a55-e415-4b13-b152-434f19924201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:41.184158 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb547a55-e415-4b13-b152-434f19924201 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:41.184436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb547a55-e415-4b13-b152-434f19924201 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:41.184707 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb547a55-e415-4b13-b152-434f19924201 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:41.202997 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:41.204958 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb547a55-e415-4b13-b152-434f19924201 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:41.206617 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb547a55-e415-4b13-b152-434f19924201 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:41.236202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-eb547a55-e415-4b13-b152-434f19924201 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:41.236202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:41.236202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:41.236202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:41.236202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:41.236202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:41.236202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:41.236202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:41.236202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:41.236202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:41.236202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:41.236202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:41.236202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:14:41.236202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:14:41.236202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:14:41.236202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:14:41.237131 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:41.237131 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:41.237131 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:41.237131 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:41.237131 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:41.237131 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:41.237131 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:41.237131 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:41.237131 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:41.237131 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:41.237131 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:41.237131 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:41.237131 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:41.237131 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:41.237131 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:41.237131 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:41.237131 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:14:41.238020 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:14:41.238020 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:14:41.238020 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:14:41.238020 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 16/31] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:41.243997 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f75ee75-874e-44ca-930c-20d4404adda2 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:41.246583 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb547a55-e415-4b13-b152-434f19924201 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:41.286737 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0df3fe-fe35-4187-8fdc-e0960104c6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:41.288236 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0df3fe-fe35-4187-8fdc-e0960104c6ce None admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:41.288520 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0df3fe-fe35-4187-8fdc-e0960104c6ce None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:41.288794 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0df3fe-fe35-4187-8fdc-e0960104c6ce None admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:41.289527 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0df3fe-fe35-4187-8fdc-e0960104c6ce None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:41.291179 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0df3fe-fe35-4187-8fdc-e0960104c6ce None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:41.321577 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 16/32] 10.222.0.30 () {60 vars in 1131 bytes} [Wed Nov 2 07:14:41 2022] POST /identity/v3/roles => generated 225 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:14:41.322840 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0df3fe-fe35-4187-8fdc-e0960104c6ce None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:41.390577 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb547a55-e415-4b13-b152-434f19924201 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:41.459620 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0df3fe-fe35-4187-8fdc-e0960104c6ce None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:41.472125 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f5a25-1fed-4744-8894-dbddb0f85b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:41.473188 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f5a25-1fed-4744-8894-dbddb0f85b1c None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:41.473608 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f5a25-1fed-4744-8894-dbddb0f85b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:41.474019 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f5a25-1fed-4744-8894-dbddb0f85b1c None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:41.474880 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f5a25-1fed-4744-8894-dbddb0f85b1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:41.475443 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f5a25-1fed-4744-8894-dbddb0f85b1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:41.482926 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f5a25-1fed-4744-8894-dbddb0f85b1c None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:41.483603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:41.505084 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a864024-7055-431e-bb07-c66565239862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:41.507734 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a864024-7055-431e-bb07-c66565239862 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:41.507734 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a864024-7055-431e-bb07-c66565239862 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:41.507734 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a864024-7055-431e-bb07-c66565239862 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:41.513100 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 17/33] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:41.519371 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f5a25-1fed-4744-8894-dbddb0f85b1c None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:41.525491 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a864024-7055-431e-bb07-c66565239862 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:41.532334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-7a864024-7055-431e-bb07-c66565239862 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:14:41.532334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:41.532334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:41.532334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:41.532334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:41.532334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:41.532334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:41.532334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:41.532334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:41.532334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:41.532334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:41.532334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:41.532334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:41.532334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:41.532334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:41.532334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:41.533243 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:41.533243 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:41.533243 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:41.533243 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:41.533243 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:41.533243 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:41.533243 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:41.533243 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:41.533243 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:41.533243 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:41.533243 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:41.533243 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:41.533243 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:41.533243 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:41.533243 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:41.533243 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:41.534415 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:41.534415 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:41.534415 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:41.534415 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:41.534415 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:41.534415 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:41.534415 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:41.534415 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:41.534415 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:41.534415 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:41.534415 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:14:41.534415 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:14:41.534415 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 17/34] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 07:14:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:41.540386 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878e219-49da-486b-b645-ae63e5bec2ca None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:41.540661 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878e219-49da-486b-b645-ae63e5bec2ca None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:41.540935 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878e219-49da-486b-b645-ae63e5bec2ca None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:41.542408 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 18/35] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:14:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:14:41.543203 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5878e219-49da-486b-b645-ae63e5bec2ca None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:41.586604 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:41.652375 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f5a25-1fed-4744-8894-dbddb0f85b1c None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:41.653437 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5878e219-49da-486b-b645-ae63e5bec2ca None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:41.687055 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81efebf-9326-4808-b98b-8b1a48a3245f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:41.687683 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81efebf-9326-4808-b98b-8b1a48a3245f None admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:41.687956 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81efebf-9326-4808-b98b-8b1a48a3245f None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:41.688220 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81efebf-9326-4808-b98b-8b1a48a3245f None admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:41.688877 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81efebf-9326-4808-b98b-8b1a48a3245f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:41.690377 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81efebf-9326-4808-b98b-8b1a48a3245f None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:41.721307 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 18/36] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Nov 2 07:14:41 2022] POST /identity/v3/projects => generated 301 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:14:41.722677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a2a9c-4afe-4038-ab01-4978d30cf464 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:41.723206 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a2a9c-4afe-4038-ab01-4978d30cf464 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:41.723206 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a2a9c-4afe-4038-ab01-4978d30cf464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:41.743477 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-636a2a9c-4afe-4038-ab01-4978d30cf464 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:41.745120 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-636a2a9c-4afe-4038-ab01-4978d30cf464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:41.801748 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7525e5-428c-4172-a85e-c2af0dd5aaba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:41.802391 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7525e5-428c-4172-a85e-c2af0dd5aaba None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:41.802689 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7525e5-428c-4172-a85e-c2af0dd5aaba None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:41.802974 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7525e5-428c-4172-a85e-c2af0dd5aaba None admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:41.803662 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7525e5-428c-4172-a85e-c2af0dd5aaba None admin] RBAC: Adding query filter params (name=admin) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:41.803971 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7525e5-428c-4172-a85e-c2af0dd5aaba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:41.805419 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7525e5-428c-4172-a85e-c2af0dd5aaba None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:41.814881 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 19/37] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:41.816346 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da160e1-0b67-4e45-b1db-304d185799ed None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:41.816955 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da160e1-0b67-4e45-b1db-304d185799ed None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:41.816955 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da160e1-0b67-4e45-b1db-304d185799ed None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:41.818314 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 20/38] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:41.824083 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c360493-83d2-48f9-a9ad-0bd13ee926bf None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:41.824400 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c360493-83d2-48f9-a9ad-0bd13ee926bf None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:41.824649 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c360493-83d2-48f9-a9ad-0bd13ee926bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:41.828403 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 19/39] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:41.829435 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a2a9c-4afe-4038-ab01-4978d30cf464 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:41.851681 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-7c360493-83d2-48f9-a9ad-0bd13ee926bf None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:41.853296 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-7c360493-83d2-48f9-a9ad-0bd13ee926bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:41.873227 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:41.900453 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:41.954392 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a2a9c-4afe-4038-ab01-4978d30cf464 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:41.955949 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 21/40] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:41.991164 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8936eb9-7b26-4218-a760-c70460850d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:41.992357 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8936eb9-7b26-4218-a760-c70460850d16 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:41.992749 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8936eb9-7b26-4218-a760-c70460850d16 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:41.993150 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8936eb9-7b26-4218-a760-c70460850d16 None admin] PATH_INFO: `/v3/users/8715d21c18b14387953f25f094b18c6d` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:41.995137 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8936eb9-7b26-4218-a760-c70460850d16 None admin] RBAC: Authorizing `identity:get_user(user_id=8715d21c18b14387953f25f094b18c6d)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:41.996854 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8936eb9-7b26-4218-a760-c70460850d16 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:41.998572 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 20/41] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:41 2022] GET /identity/v3/users/8715d21c18b14387953f25f094b18c6d => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:42.005724 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c360493-83d2-48f9-a9ad-0bd13ee926bf None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:42.091948 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8936eb9-7b26-4218-a760-c70460850d16 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:42.108976 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c360493-83d2-48f9-a9ad-0bd13ee926bf None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:42.145368 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e2dda-e339-41c6-9ee8-8bc52bbc5e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:42.145996 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e2dda-e339-41c6-9ee8-8bc52bbc5e18 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:42.146273 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e2dda-e339-41c6-9ee8-8bc52bbc5e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:42.146526 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e2dda-e339-41c6-9ee8-8bc52bbc5e18 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:42.147779 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00e2dda-e339-41c6-9ee8-8bc52bbc5e18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:42.149615 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00e2dda-e339-41c6-9ee8-8bc52bbc5e18 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:42.156254 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 22/42] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:42.157303 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e2dda-e339-41c6-9ee8-8bc52bbc5e18 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:42.196694 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8936eb9-7b26-4218-a760-c70460850d16 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:42.198194 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:42.201609 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:42.234975 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae270949-af12-43b9-9adb-52a560315a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:42.235590 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae270949-af12-43b9-9adb-52a560315a49 None admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:42.235860 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae270949-af12-43b9-9adb-52a560315a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:42.236129 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae270949-af12-43b9-9adb-52a560315a49 None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:42.236798 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae270949-af12-43b9-9adb-52a560315a49 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:42.238276 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae270949-af12-43b9-9adb-52a560315a49 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:42.254268 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 21/43] 10.222.0.30 () {60 vars in 1131 bytes} [Wed Nov 2 07:14:42 2022] POST /identity/v3/roles => generated 223 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:14:42.255205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae270949-af12-43b9-9adb-52a560315a49 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:42.272865 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e2dda-e339-41c6-9ee8-8bc52bbc5e18 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:42.314345 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c225a-8d36-4b15-bc90-eae64ea9011a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:42.315032 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c225a-8d36-4b15-bc90-eae64ea9011a None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:42.315320 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c225a-8d36-4b15-bc90-eae64ea9011a None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:42.315702 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c225a-8d36-4b15-bc90-eae64ea9011a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:42.317079 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6c225a-8d36-4b15-bc90-eae64ea9011a None admin] RBAC: Adding query filter params (role.id=1e35bb725eb8475ea63e2d9538ab67d5, scope.domain.id=default, user.id=8715d21c18b14387953f25f094b18c6d) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:42.317210 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6c225a-8d36-4b15-bc90-eae64ea9011a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:42.318786 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6c225a-8d36-4b15-bc90-eae64ea9011a None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:42.338708 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 23/44] 10.222.0.30 () {56 vars in 1322 bytes} [Wed Nov 2 07:14:42 2022] GET /identity/v3/role_assignments?user.id=8715d21c18b14387953f25f094b18c6d&scope.domain.id=default&role.id=1e35bb725eb8475ea63e2d9538ab67d5 => generated 530 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:42.368978 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae270949-af12-43b9-9adb-52a560315a49 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:42.410704 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1cb48-a43d-4b26-93f1-0e7ce299cdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:42.411428 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1cb48-a43d-4b26-93f1-0e7ce299cdae None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:42.411663 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1cb48-a43d-4b26-93f1-0e7ce299cdae None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:42.412015 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1cb48-a43d-4b26-93f1-0e7ce299cdae None admin] PATH_INFO: `/v3/domains/default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:42.413375 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d1cb48-a43d-4b26-93f1-0e7ce299cdae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:42.414910 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d1cb48-a43d-4b26-93f1-0e7ce299cdae None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:42.416898 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 22/45] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:42.422971 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c225a-8d36-4b15-bc90-eae64ea9011a None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:42.541614 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c225a-8d36-4b15-bc90-eae64ea9011a None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:42.545838 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:42.578687 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f9a0ad-3bce-404e-8d49-0b2bbfce2830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:42.579364 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f9a0ad-3bce-404e-8d49-0b2bbfce2830 None admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:42.579659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f9a0ad-3bce-404e-8d49-0b2bbfce2830 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:42.579939 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f9a0ad-3bce-404e-8d49-0b2bbfce2830 None admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:42.580600 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f9a0ad-3bce-404e-8d49-0b2bbfce2830 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:42.582099 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f9a0ad-3bce-404e-8d49-0b2bbfce2830 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:42.612637 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 24/46] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Nov 2 07:14:42 2022] POST /identity/v3/projects => generated 312 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:14:44.760430 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7a4c1-edcd-4ad3-9429-674541aa969f None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:44.761268 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7a4c1-edcd-4ad3-9429-674541aa969f None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:44.761268 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7a4c1-edcd-4ad3-9429-674541aa969f None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:44.763469 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 23/47] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:14:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:44.768963 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b7ead-445a-4055-b205-42f220ce1cca None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:44.769378 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b7ead-445a-4055-b205-42f220ce1cca None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:44.769511 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b7ead-445a-4055-b205-42f220ce1cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:44.790034 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-2c1b7ead-445a-4055-b205-42f220ce1cca None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:44.791254 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-2c1b7ead-445a-4055-b205-42f220ce1cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:44.870600 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 25/48] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:45.139743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7a4c1-edcd-4ad3-9429-674541aa969f None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:45.176189 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:45.252302 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7a4c1-edcd-4ad3-9429-674541aa969f None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:45.288869 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eed39e3-875a-45d3-93e4-3e6c796f67c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:45.289491 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed39e3-875a-45d3-93e4-3e6c796f67c7 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:45.289774 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed39e3-875a-45d3-93e4-3e6c796f67c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:45.290038 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed39e3-875a-45d3-93e4-3e6c796f67c7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:45.291297 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eed39e3-875a-45d3-93e4-3e6c796f67c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:45.293032 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eed39e3-875a-45d3-93e4-3e6c796f67c7 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:45.294590 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 24/49] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:45.301203 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b7ead-445a-4055-b205-42f220ce1cca None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:45.395420 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:45.411819 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b7ead-445a-4055-b205-42f220ce1cca None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:45.446326 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa9733-2ed4-42da-8525-2e5f1b9e4276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:45.446961 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa9733-2ed4-42da-8525-2e5f1b9e4276 None admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:45.447234 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa9733-2ed4-42da-8525-2e5f1b9e4276 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:45.447505 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa9733-2ed4-42da-8525-2e5f1b9e4276 None admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:45.448139 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa9733-2ed4-42da-8525-2e5f1b9e4276 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:45.449583 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa9733-2ed4-42da-8525-2e5f1b9e4276 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:45.478579 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 26/50] 10.222.0.30 () {60 vars in 1140 bytes} [Wed Nov 2 07:14:45 2022] POST /identity/v3/projects => generated 298 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:14:47.637449 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5d1cd-87d1-404c-8fac-d51507272ef0 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:47.637914 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5d1cd-87d1-404c-8fac-d51507272ef0 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:47.638026 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5d1cd-87d1-404c-8fac-d51507272ef0 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:47.639497 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 25/51] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:47.645355 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c6e0c5-24ae-498f-9db9-ba84c1a15302 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:47.645759 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c6e0c5-24ae-498f-9db9-ba84c1a15302 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:47.646079 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c6e0c5-24ae-498f-9db9-ba84c1a15302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:47.667599 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-77c6e0c5-24ae-498f-9db9-ba84c1a15302 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:47.668778 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-77c6e0c5-24ae-498f-9db9-ba84c1a15302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:47.738329 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 27/52] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:48.002185 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c5d1cd-87d1-404c-8fac-d51507272ef0 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:48.109897 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:48.115345 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c5d1cd-87d1-404c-8fac-d51507272ef0 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:48.150307 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a04c1bf-e137-4b90-a566-64e2c94c562b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:48.150959 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04c1bf-e137-4b90-a566-64e2c94c562b None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:48.151235 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04c1bf-e137-4b90-a566-64e2c94c562b None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:48.151506 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04c1bf-e137-4b90-a566-64e2c94c562b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:48.152620 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a04c1bf-e137-4b90-a566-64e2c94c562b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:48.154070 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a04c1bf-e137-4b90-a566-64e2c94c562b None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:48.155674 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 26/53] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:48.162042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c6e0c5-24ae-498f-9db9-ba84c1a15302 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:48.245726 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:48.283463 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c6e0c5-24ae-498f-9db9-ba84c1a15302 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:48.320492 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95f544-0b08-4c87-ab0b-81770ca7fe47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:48.321099 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95f544-0b08-4c87-ab0b-81770ca7fe47 None admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:48.321354 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95f544-0b08-4c87-ab0b-81770ca7fe47 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:48.321627 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95f544-0b08-4c87-ab0b-81770ca7fe47 None admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:48.322268 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d95f544-0b08-4c87-ab0b-81770ca7fe47 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:48.324049 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d95f544-0b08-4c87-ab0b-81770ca7fe47 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:48.353976 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 28/54] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 07:14:48 2022] POST /identity/v3/users => generated 275 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:14:51.506372 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17de7fd-cf66-4dda-8c8e-ad3ad807c61f None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:51.507562 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17de7fd-cf66-4dda-8c8e-ad3ad807c61f None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:51.508112 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17de7fd-cf66-4dda-8c8e-ad3ad807c61f None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:51.509661 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 27/55] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:14:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:51.515784 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057212b-f48c-45d6-8933-b72ed310d980 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:51.516674 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057212b-f48c-45d6-8933-b72ed310d980 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:51.517159 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057212b-f48c-45d6-8933-b72ed310d980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:51.547700 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-f057212b-f48c-45d6-8933-b72ed310d980 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:51.548917 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-f057212b-f48c-45d6-8933-b72ed310d980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:51.651545 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 29/56] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:51.729748 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c4f43-8533-4e95-b733-0e4aad5f79ce None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:51.730390 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c4f43-8533-4e95-b733-0e4aad5f79ce None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:51.730876 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c4f43-8533-4e95-b733-0e4aad5f79ce None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:51.732420 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 28/57] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:14:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:51.754247 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74d8d8-a051-4992-887e-b1787bc850bc None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:51.754520 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74d8d8-a051-4992-887e-b1787bc850bc None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:51.754820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74d8d8-a051-4992-887e-b1787bc850bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:51.783705 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-4c74d8d8-a051-4992-887e-b1787bc850bc None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:51.784830 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-4c74d8d8-a051-4992-887e-b1787bc850bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:51.850165 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:51.876354 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 30/58] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:52.036177 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c4f43-8533-4e95-b733-0e4aad5f79ce None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:52.058331 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9b472-8427-47fd-a9e1-c1afa12023b9 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:52.058642 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9b472-8427-47fd-a9e1-c1afa12023b9 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:52.058893 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9b472-8427-47fd-a9e1-c1afa12023b9 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:52.060508 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 31/59] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:14:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:52.065806 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b6fa6-d176-4aac-964c-23dce1694d0a None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:52.066084 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b6fa6-d176-4aac-964c-23dce1694d0a None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:52.066353 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b6fa6-d176-4aac-964c-23dce1694d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:52.103331 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-b14b6fa6-d176-4aac-964c-23dce1694d0a None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:52.104890 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-b14b6fa6-d176-4aac-964c-23dce1694d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:52.167398 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c4f43-8533-4e95-b733-0e4aad5f79ce None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:52.202765 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 32/60] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:14:52.202765 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e37f8-6669-4b32-a7ba-8e9285e36bb6 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:52.202765 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e37f8-6669-4b32-a7ba-8e9285e36bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:52.202765 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e37f8-6669-4b32-a7ba-8e9285e36bb6 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:52.203565 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 33/61] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:14:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:52.204396 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e37f8-6669-4b32-a7ba-8e9285e36bb6 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:52.213246 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a8adc-030d-4ee4-9787-7389ca185e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:52.214082 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a8adc-030d-4ee4-9787-7389ca185e66 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:52.214506 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a8adc-030d-4ee4-9787-7389ca185e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:52.214949 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a8adc-030d-4ee4-9787-7389ca185e66 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:52.230036 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a8adc-030d-4ee4-9787-7389ca185e66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:52.237993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-d25a8adc-030d-4ee4-9787-7389ca185e66 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:14:52.237993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:52.237993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:52.237993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:52.237993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:52.237993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:52.237993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:52.237993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:52.237993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:52.237993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:52.237993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:52.237993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:52.237993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:52.237993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:52.237993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:52.237993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:52.239066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:52.239066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:52.239066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:52.239066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:52.239066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:52.239066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:52.239066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:52.239066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:52.239066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:52.239066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:52.239066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:52.239066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:52.239066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:52.239066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:52.239066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:52.239066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:52.239993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:52.239993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:52.239993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:52.239993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:52.239993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:52.239993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:52.239993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:52.239993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:52.239993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:52.239993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:52.239993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:14:52.239993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:14:52.242915 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 29/62] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:52.244489 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428754b6-217e-4814-9b7d-b1bdb60b0b91 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:52.244905 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428754b6-217e-4814-9b7d-b1bdb60b0b91 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:52.245306 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428754b6-217e-4814-9b7d-b1bdb60b0b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:52.269334 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-428754b6-217e-4814-9b7d-b1bdb60b0b91 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:52.270811 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-428754b6-217e-4814-9b7d-b1bdb60b0b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:52.297046 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:52.331589 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e37f8-6669-4b32-a7ba-8e9285e36bb6 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:52.350828 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 30/63] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:52.352101 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428754b6-217e-4814-9b7d-b1bdb60b0b91 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:52.359063 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:52.396807 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96634228-d372-4db2-a89e-94d696c9922e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:52.398740 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96634228-d372-4db2-a89e-94d696c9922e None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:52.399032 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96634228-d372-4db2-a89e-94d696c9922e None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:52.399311 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96634228-d372-4db2-a89e-94d696c9922e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:52.423531 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96634228-d372-4db2-a89e-94d696c9922e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:52.437437 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-96634228-d372-4db2-a89e-94d696c9922e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:14:52.437437 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:52.437437 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:52.437437 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:52.437437 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:52.437437 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:52.437437 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:52.437437 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:52.437437 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:52.437437 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:52.437437 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:52.437437 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:52.437437 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:52.437437 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:52.437437 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:52.437437 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:52.438517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:52.438517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:52.438517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:52.438517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:52.438517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:52.438517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:52.438517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:52.438517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:52.438517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:52.438517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:52.438517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:52.438517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:52.438517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:52.438517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:52.438517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:52.438517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:52.439430 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:52.439430 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:52.439430 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:52.439430 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:52.439430 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:52.439430 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:52.439430 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:52.439430 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:52.439430 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:52.439430 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:52.439430 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:14:52.439430 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:14:52.441865 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 34/64] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 07:14:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:52.452615 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96634228-d372-4db2-a89e-94d696c9922e None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:52.578865 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96634228-d372-4db2-a89e-94d696c9922e None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:52.614557 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5213263-8677-4a3f-8375-eb64118241cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:52.615191 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5213263-8677-4a3f-8375-eb64118241cd None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:52.615475 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5213263-8677-4a3f-8375-eb64118241cd None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:52.615740 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5213263-8677-4a3f-8375-eb64118241cd None admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:52.616634 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5213263-8677-4a3f-8375-eb64118241cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:52.616938 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5213263-8677-4a3f-8375-eb64118241cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:52.618335 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5213263-8677-4a3f-8375-eb64118241cd None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:52.627443 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 35/65] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:14:52.628209 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428754b6-217e-4814-9b7d-b1bdb60b0b91 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:52.628855 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5213263-8677-4a3f-8375-eb64118241cd None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:52.667975 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:52.669151 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf9315-51e7-4a80-99b8-b6e957999424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:52.669743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf9315-51e7-4a80-99b8-b6e957999424 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:52.670017 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf9315-51e7-4a80-99b8-b6e957999424 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:52.670289 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf9315-51e7-4a80-99b8-b6e957999424 None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:52.670992 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf9315-51e7-4a80-99b8-b6e957999424 None admin] RBAC: Adding query filter params (name=member) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:52.671293 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf9315-51e7-4a80-99b8-b6e957999424 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:52.672693 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf9315-51e7-4a80-99b8-b6e957999424 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:52.684874 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 31/66] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 07:14:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:52.686223 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01b68e-7874-481a-9892-4cd41cf9dc63 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:52.686505 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01b68e-7874-481a-9892-4cd41cf9dc63 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:52.686807 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01b68e-7874-481a-9892-4cd41cf9dc63 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:52.688010 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 32/67] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:14:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:52.688700 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01b68e-7874-481a-9892-4cd41cf9dc63 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:52.707361 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:52.737378 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5213263-8677-4a3f-8375-eb64118241cd None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:52.772270 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48bc1f7-bdc2-4d3d-8950-b99cc6c1a7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:52.772978 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bc1f7-bdc2-4d3d-8950-b99cc6c1a7bb None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:52.773263 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bc1f7-bdc2-4d3d-8950-b99cc6c1a7bb None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:52.773534 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bc1f7-bdc2-4d3d-8950-b99cc6c1a7bb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:52.784020 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48bc1f7-bdc2-4d3d-8950-b99cc6c1a7bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:52.788914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-c48bc1f7-bdc2-4d3d-8950-b99cc6c1a7bb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:14:52.788914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:52.788914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:52.788914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:52.788914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:52.788914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:52.788914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:52.788914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:52.788914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:52.788914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:52.788914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:52.788914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:52.788914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:52.788914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:52.788914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:52.788914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:52.789769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:52.789769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:52.789769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:52.789769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:52.789769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:52.789769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:52.789769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:52.789769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:52.789769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:52.789769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:52.789769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:52.789769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:52.789769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:52.789769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:52.789769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:52.789769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:52.791777 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:52.791777 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:52.791777 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:52.791777 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:52.791777 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:52.791777 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:52.791777 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:52.791777 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:52.791777 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:52.791777 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:52.791777 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:14:52.791777 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:14:52.791777 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 36/68] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:52.791777 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48bc1f7-bdc2-4d3d-8950-b99cc6c1a7bb None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:52.792929 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01b68e-7874-481a-9892-4cd41cf9dc63 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:52.829216 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e05278-6a97-40c8-a463-20cbf3c3bb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:52.829777 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e05278-6a97-40c8-a463-20cbf3c3bb1e None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:52.830057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e05278-6a97-40c8-a463-20cbf3c3bb1e None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:52.830330 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e05278-6a97-40c8-a463-20cbf3c3bb1e None admin] PATH_INFO: `/v3/users/8715d21c18b14387953f25f094b18c6d` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:52.831910 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e05278-6a97-40c8-a463-20cbf3c3bb1e None admin] RBAC: Authorizing `identity:get_user(user_id=8715d21c18b14387953f25f094b18c6d)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:52.833729 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e05278-6a97-40c8-a463-20cbf3c3bb1e None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:52.836202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 33/69] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:52 2022] GET /identity/v3/users/8715d21c18b14387953f25f094b18c6d => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:52.836974 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e05278-6a97-40c8-a463-20cbf3c3bb1e None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:52.885197 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:52.891442 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48bc1f7-bdc2-4d3d-8950-b99cc6c1a7bb None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:52.923843 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:52.925191 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b378e71a-3dc6-4129-b9d8-bb7744799566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:52.925828 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378e71a-3dc6-4129-b9d8-bb7744799566 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:52.926115 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378e71a-3dc6-4129-b9d8-bb7744799566 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:52.926377 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378e71a-3dc6-4129-b9d8-bb7744799566 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:52.937202 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b378e71a-3dc6-4129-b9d8-bb7744799566 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:52.939593 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e05278-6a97-40c8-a463-20cbf3c3bb1e None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:52.942019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-b378e71a-3dc6-4129-b9d8-bb7744799566 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:14:52.942019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:52.942019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:52.942019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:52.942019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:52.942019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:52.942019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:52.942019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:52.942019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:52.942019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:52.942019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:52.942019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:52.942019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:52.942019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:52.942019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:52.942019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:52.942960 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:52.942960 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:52.942960 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:52.942960 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:52.942960 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:52.942960 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:52.942960 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:52.942960 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:52.942960 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:52.942960 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:52.942960 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:52.942960 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:52.942960 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:52.942960 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:52.942960 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:52.942960 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:52.943695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:52.943695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:52.943695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:52.943695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:52.943695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:52.943695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:52.943695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:52.943695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:52.943695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:52.943695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:52.943695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:14:52.943695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:14:52.943695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 37/70] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Nov 2 07:14:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:52.944898 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e738a-18d7-44f7-8195-35c3e737759b None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:52.944898 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e738a-18d7-44f7-8195-35c3e737759b None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:52.944898 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e738a-18d7-44f7-8195-35c3e737759b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:52.962361 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-444e738a-18d7-44f7-8195-35c3e737759b None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:52.963599 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-444e738a-18d7-44f7-8195-35c3e737759b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:52.974959 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba4d57-39ea-4787-94d6-d082393437b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:52.975660 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba4d57-39ea-4787-94d6-d082393437b8 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:52.975938 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba4d57-39ea-4787-94d6-d082393437b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:52.976206 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba4d57-39ea-4787-94d6-d082393437b8 None admin] PATH_INFO: `/v3/users/dc54984fb77c49da93c625769084ee80` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:52.990557 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba4d57-39ea-4787-94d6-d082393437b8 None admin] RBAC: Authorizing `identity:get_user(user_id=dc54984fb77c49da93c625769084ee80)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:52.992065 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba4d57-39ea-4787-94d6-d082393437b8 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:52.993679 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 34/71] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:52 2022] GET /identity/v3/users/dc54984fb77c49da93c625769084ee80 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:52.994441 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba4d57-39ea-4787-94d6-d082393437b8 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:53.034613 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 38/72] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:53.035747 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e738a-18d7-44f7-8195-35c3e737759b None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:53.104776 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba4d57-39ea-4787-94d6-d082393437b8 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:53.128824 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:53.143512 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b4ec84-a911-49b4-9c8d-258bd078a274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:53.144127 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4ec84-a911-49b4-9c8d-258bd078a274 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:53.144403 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4ec84-a911-49b4-9c8d-258bd078a274 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:53.144672 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4ec84-a911-49b4-9c8d-258bd078a274 None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:53.144927 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e738a-18d7-44f7-8195-35c3e737759b None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:53.145330 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b4ec84-a911-49b4-9c8d-258bd078a274 None admin] RBAC: Adding query filter params (name=member) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:53.145626 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b4ec84-a911-49b4-9c8d-258bd078a274 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:53.147032 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b4ec84-a911-49b4-9c8d-258bd078a274 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:53.156336 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 35/73] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 07:14:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:53.156943 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b4ec84-a911-49b4-9c8d-258bd078a274 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:53.165966 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:53.184959 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bdb25a-e7c9-42cf-94ba-1d5876cbc624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:53.185696 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bdb25a-e7c9-42cf-94ba-1d5876cbc624 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:53.185882 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bdb25a-e7c9-42cf-94ba-1d5876cbc624 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:53.186168 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bdb25a-e7c9-42cf-94ba-1d5876cbc624 None admin] PATH_INFO: `/v3/projects/03fe633d52c14c2aa697f9116b41facc` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:53.187304 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bdb25a-e7c9-42cf-94ba-1d5876cbc624 None admin] RBAC: Authorizing `identity:get_project(project_id=03fe633d52c14c2aa697f9116b41facc)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:53.188772 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bdb25a-e7c9-42cf-94ba-1d5876cbc624 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:53.194741 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 39/74] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 07:14:53 2022] GET /identity/v3/projects/03fe633d52c14c2aa697f9116b41facc => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:53.194741 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bdb25a-e7c9-42cf-94ba-1d5876cbc624 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:53.264443 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b4ec84-a911-49b4-9c8d-258bd078a274 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:53.299438 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bdb25a-e7c9-42cf-94ba-1d5876cbc624 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:53.301744 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd27ba-b07a-4735-92dc-be86d4d29922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:53.302354 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd27ba-b07a-4735-92dc-be86d4d29922 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:53.302630 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd27ba-b07a-4735-92dc-be86d4d29922 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:53.302998 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd27ba-b07a-4735-92dc-be86d4d29922 None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:53.303689 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdd27ba-b07a-4735-92dc-be86d4d29922 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:53.304013 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdd27ba-b07a-4735-92dc-be86d4d29922 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:53.305382 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdd27ba-b07a-4735-92dc-be86d4d29922 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:53.314539 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 36/75] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:14:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:53.315356 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd27ba-b07a-4735-92dc-be86d4d29922 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:53.337285 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e105d-b313-45ec-9252-b008c1ebefc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:53.337925 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e105d-b313-45ec-9252-b008c1ebefc6 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:53.338203 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e105d-b313-45ec-9252-b008c1ebefc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:53.338476 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e105d-b313-45ec-9252-b008c1ebefc6 None admin] PATH_INFO: `/v3/projects/03fe633d52c14c2aa697f9116b41facc` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:53.339644 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0e105d-b313-45ec-9252-b008c1ebefc6 None admin] RBAC: Authorizing `identity:get_project(project_id=03fe633d52c14c2aa697f9116b41facc)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:53.341111 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0e105d-b313-45ec-9252-b008c1ebefc6 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:53.347549 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 40/76] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 07:14:53 2022] GET /identity/v3/projects/03fe633d52c14c2aa697f9116b41facc => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:53.347549 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e105d-b313-45ec-9252-b008c1ebefc6 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:53.371240 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:53.381985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:53.416452 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd27ba-b07a-4735-92dc-be86d4d29922 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:53.445792 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e105d-b313-45ec-9252-b008c1ebefc6 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:53.448860 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d8fae-6275-4f5a-a0f7-62e72a6e8c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:53.449493 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d8fae-6275-4f5a-a0f7-62e72a6e8c94 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:53.449769 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d8fae-6275-4f5a-a0f7-62e72a6e8c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:53.450041 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d8fae-6275-4f5a-a0f7-62e72a6e8c94 None admin] PATH_INFO: `/v3/users/dc54984fb77c49da93c625769084ee80` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:53.451246 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d8fae-6275-4f5a-a0f7-62e72a6e8c94 None admin] RBAC: Authorizing `identity:get_user(user_id=dc54984fb77c49da93c625769084ee80)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:53.452738 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d8fae-6275-4f5a-a0f7-62e72a6e8c94 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:53.454302 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 37/77] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:53 2022] GET /identity/v3/users/dc54984fb77c49da93c625769084ee80 => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:53.459582 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d8fae-6275-4f5a-a0f7-62e72a6e8c94 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:53.479123 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5b6e9-955a-4a29-a2d0-1da9c1d83d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:53.479783 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5b6e9-955a-4a29-a2d0-1da9c1d83d18 None admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:53.480060 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5b6e9-955a-4a29-a2d0-1da9c1d83d18 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:53.480333 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5b6e9-955a-4a29-a2d0-1da9c1d83d18 None admin] PATH_INFO: `/v3/projects/03fe633d52c14c2aa697f9116b41facc/users/8715d21c18b14387953f25f094b18c6d/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:53.482199 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e5b6e9-955a-4a29-a2d0-1da9c1d83d18 None admin] RBAC: Authorizing `identity:create_grant(project_id=03fe633d52c14c2aa697f9116b41facc, user_id=8715d21c18b14387953f25f094b18c6d, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:53.483676 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e5b6e9-955a-4a29-a2d0-1da9c1d83d18 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:53.497311 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 41/78] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 07:14:53 2022] PUT /identity/v3/projects/03fe633d52c14c2aa697f9116b41facc/users/8715d21c18b14387953f25f094b18c6d/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:14:53.498254 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5b6e9-955a-4a29-a2d0-1da9c1d83d18 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:53.567434 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d8fae-6275-4f5a-a0f7-62e72a6e8c94 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:53.593701 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:53.615680 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e57fa-9b92-4e99-870f-61740d246b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:53.616516 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e57fa-9b92-4e99-870f-61740d246b87 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:53.616792 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e57fa-9b92-4e99-870f-61740d246b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:53.617061 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e57fa-9b92-4e99-870f-61740d246b87 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:53.618250 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e57fa-9b92-4e99-870f-61740d246b87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:53.619756 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e57fa-9b92-4e99-870f-61740d246b87 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:53.620054 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5b6e9-955a-4a29-a2d0-1da9c1d83d18 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:53.621459 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 38/79] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:53.622164 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e57fa-9b92-4e99-870f-61740d246b87 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:53.626624 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:53.658386 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dee55a-e2c6-485d-9b2d-a1d9f8b3a1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:53.659065 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dee55a-e2c6-485d-9b2d-a1d9f8b3a1ec None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:53.659345 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dee55a-e2c6-485d-9b2d-a1d9f8b3a1ec None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:53.659880 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dee55a-e2c6-485d-9b2d-a1d9f8b3a1ec None admin] PATH_INFO: `/v3/users/dc54984fb77c49da93c625769084ee80` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:53.660789 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dee55a-e2c6-485d-9b2d-a1d9f8b3a1ec None admin] RBAC: Authorizing `identity:get_user(user_id=dc54984fb77c49da93c625769084ee80)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:53.662257 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dee55a-e2c6-485d-9b2d-a1d9f8b3a1ec None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:53.664760 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 42/80] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:53 2022] GET /identity/v3/users/dc54984fb77c49da93c625769084ee80 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:53.665280 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dee55a-e2c6-485d-9b2d-a1d9f8b3a1ec None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:53.743226 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e57fa-9b92-4e99-870f-61740d246b87 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:53.769824 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dee55a-e2c6-485d-9b2d-a1d9f8b3a1ec None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:53.777914 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bff771-7d83-4fb4-ad3b-2c5573358635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:53.778612 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bff771-7d83-4fb4-ad3b-2c5573358635 None admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:53.778894 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bff771-7d83-4fb4-ad3b-2c5573358635 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:53.779181 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bff771-7d83-4fb4-ad3b-2c5573358635 None admin] PATH_INFO: `/v3/projects/03fe633d52c14c2aa697f9116b41facc/users/dc54984fb77c49da93c625769084ee80/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:53.781156 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bff771-7d83-4fb4-ad3b-2c5573358635 None admin] RBAC: Authorizing `identity:create_grant(project_id=03fe633d52c14c2aa697f9116b41facc, user_id=dc54984fb77c49da93c625769084ee80, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:53.782636 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bff771-7d83-4fb4-ad3b-2c5573358635 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:53.806697 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 39/81] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 07:14:53 2022] PUT /identity/v3/projects/03fe633d52c14c2aa697f9116b41facc/users/dc54984fb77c49da93c625769084ee80/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:14:53.806697 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bff771-7d83-4fb4-ad3b-2c5573358635 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:53.816267 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd132f-f69f-4dbb-914d-bab0d00263aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:53.817045 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd132f-f69f-4dbb-914d-bab0d00263aa None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:53.817408 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd132f-f69f-4dbb-914d-bab0d00263aa None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:53.817758 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd132f-f69f-4dbb-914d-bab0d00263aa None admin] PATH_INFO: `/v3/projects/60c61605c0a6402485c7e69d18304dba` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:53.819158 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd132f-f69f-4dbb-914d-bab0d00263aa None admin] RBAC: Authorizing `identity:get_project(project_id=60c61605c0a6402485c7e69d18304dba)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:53.820875 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd132f-f69f-4dbb-914d-bab0d00263aa None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:53.827486 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 43/82] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 07:14:53 2022] GET /identity/v3/projects/60c61605c0a6402485c7e69d18304dba => generated 312 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:53.827486 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd132f-f69f-4dbb-914d-bab0d00263aa None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:53.847885 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:53.901505 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:53.948819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bff771-7d83-4fb4-ad3b-2c5573358635 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:53.955849 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd132f-f69f-4dbb-914d-bab0d00263aa None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:53.992580 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c6679-9720-4f94-b1d9-2633f2ed24ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:53.993222 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c6679-9720-4f94-b1d9-2633f2ed24ef None admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:53.993503 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c6679-9720-4f94-b1d9-2633f2ed24ef None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:53.993777 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c6679-9720-4f94-b1d9-2633f2ed24ef None admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:53.994444 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c6679-9720-4f94-b1d9-2633f2ed24ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:53.994877 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2654133e-13d4-44e6-b058-27b4a986a4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:53.995527 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654133e-13d4-44e6-b058-27b4a986a4af None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:53.995806 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654133e-13d4-44e6-b058-27b4a986a4af None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:53.996071 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654133e-13d4-44e6-b058-27b4a986a4af None admin] PATH_INFO: `/v3/projects/03fe633d52c14c2aa697f9116b41facc` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:53.996163 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c6679-9720-4f94-b1d9-2633f2ed24ef None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:53.997198 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2654133e-13d4-44e6-b058-27b4a986a4af None admin] RBAC: Authorizing `identity:get_project(project_id=03fe633d52c14c2aa697f9116b41facc)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:53.998648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2654133e-13d4-44e6-b058-27b4a986a4af None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:54.000481 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 44/83] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 07:14:53 2022] GET /identity/v3/projects/03fe633d52c14c2aa697f9116b41facc => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:54.001052 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2654133e-13d4-44e6-b058-27b4a986a4af None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:54.028101 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 40/84] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 07:14:53 2022] POST /identity/v3/users => generated 289 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:14:54.029462 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c6679-9720-4f94-b1d9-2633f2ed24ef None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:54.102309 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:54.127951 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2654133e-13d4-44e6-b058-27b4a986a4af None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:54.181810 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c604a-7be4-41c2-ab86-c3601b5a157f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:54.182418 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c604a-7be4-41c2-ab86-c3601b5a157f None admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:54.182717 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c604a-7be4-41c2-ab86-c3601b5a157f None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:54.182997 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c604a-7be4-41c2-ab86-c3601b5a157f None admin] PATH_INFO: `/v3/projects/60c61605c0a6402485c7e69d18304dba/users/dc54984fb77c49da93c625769084ee80/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:54.185782 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c604a-7be4-41c2-ab86-c3601b5a157f None admin] RBAC: Authorizing `identity:create_grant(project_id=60c61605c0a6402485c7e69d18304dba, user_id=dc54984fb77c49da93c625769084ee80, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:54.187319 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c604a-7be4-41c2-ab86-c3601b5a157f None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:54.203442 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c6679-9720-4f94-b1d9-2633f2ed24ef None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:54.215015 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 45/85] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 07:14:54 2022] PUT /identity/v3/projects/60c61605c0a6402485c7e69d18304dba/users/dc54984fb77c49da93c625769084ee80/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:14:54.243069 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:54.260356 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1054c030-93c9-42c1-bd7d-de67e7d9c826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:54.261228 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054c030-93c9-42c1-bd7d-de67e7d9c826 None admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:54.261860 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054c030-93c9-42c1-bd7d-de67e7d9c826 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:54.262283 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054c030-93c9-42c1-bd7d-de67e7d9c826 None admin] PATH_INFO: `/v3/projects/03fe633d52c14c2aa697f9116b41facc/users/dc54984fb77c49da93c625769084ee80/roles/39eab44f330440ebbefe39e538462165` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:54.265647 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1054c030-93c9-42c1-bd7d-de67e7d9c826 None admin] RBAC: Authorizing `identity:create_grant(project_id=03fe633d52c14c2aa697f9116b41facc, user_id=dc54984fb77c49da93c625769084ee80, role_id=39eab44f330440ebbefe39e538462165)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:54.267501 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1054c030-93c9-42c1-bd7d-de67e7d9c826 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:54.289654 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 41/86] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 07:14:54 2022] PUT /identity/v3/projects/03fe633d52c14c2aa697f9116b41facc/users/dc54984fb77c49da93c625769084ee80/roles/39eab44f330440ebbefe39e538462165 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:14:55.971242 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4572e3-b316-458c-9525-a533ba99ab61 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:55.971775 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4572e3-b316-458c-9525-a533ba99ab61 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:55.971775 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4572e3-b316-458c-9525-a533ba99ab61 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:55.973078 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 46/87] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:55.978495 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0092bb-0aed-4fa1-bf48-9cb0659a0174 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:55.978911 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0092bb-0aed-4fa1-bf48-9cb0659a0174 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:55.979089 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0092bb-0aed-4fa1-bf48-9cb0659a0174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:56.004464 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-3b0092bb-0aed-4fa1-bf48-9cb0659a0174 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:56.005748 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-3b0092bb-0aed-4fa1-bf48-9cb0659a0174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:56.090150 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 42/88] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:56.622198 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4572e3-b316-458c-9525-a533ba99ab61 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:56.764277 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4572e3-b316-458c-9525-a533ba99ab61 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:56.805991 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d63141-d6d1-400d-ac0d-562b1e309140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:56.807005 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d63141-d6d1-400d-ac0d-562b1e309140 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:56.807559 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d63141-d6d1-400d-ac0d-562b1e309140 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:56.808024 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d63141-d6d1-400d-ac0d-562b1e309140 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:56.830263 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d63141-d6d1-400d-ac0d-562b1e309140 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:56.835426 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-f3d63141-d6d1-400d-ac0d-562b1e309140 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:14:56.835426 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:56.835426 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:56.835426 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:56.835426 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:56.835426 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:56.835426 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:56.835426 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:56.835426 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:56.835426 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:56.835426 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:56.835426 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:56.835426 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:56.835426 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:56.835426 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:56.835426 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:56.836420 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:56.836420 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:56.836420 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:56.836420 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:56.836420 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:56.836420 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:56.836420 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:56.836420 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:56.836420 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:56.836420 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:56.836420 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:56.836420 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:56.836420 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:56.836420 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:56.836420 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:56.836420 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:56.837195 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:56.837195 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:56.837195 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:56.837195 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:56.837195 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:56.837195 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:56.837195 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:56.837195 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:56.837195 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:56.837195 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:56.837195 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:14:56.837195 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:14:56.839399 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 47/89] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 07:14:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:56.845413 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0092bb-0aed-4fa1-bf48-9cb0659a0174 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:56.986501 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0092bb-0aed-4fa1-bf48-9cb0659a0174 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:57.004731 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:57.037557 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f3c13-e2f5-47f5-88e7-47e082c865cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:57.038192 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f3c13-e2f5-47f5-88e7-47e082c865cf None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:57.038471 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f3c13-e2f5-47f5-88e7-47e082c865cf None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:57.038896 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f3c13-e2f5-47f5-88e7-47e082c865cf None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:57.039840 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f3c13-e2f5-47f5-88e7-47e082c865cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:57.040147 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f3c13-e2f5-47f5-88e7-47e082c865cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:57.041586 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f3c13-e2f5-47f5-88e7-47e082c865cf None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:57.055606 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 43/90] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:57.061510 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d63141-d6d1-400d-ac0d-562b1e309140 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:57.078185 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:57.099473 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93492867-ff00-4fe5-a3b2-d2db083aeb63 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:57.099779 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93492867-ff00-4fe5-a3b2-d2db083aeb63 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:57.100113 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93492867-ff00-4fe5-a3b2-d2db083aeb63 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:57.107041 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 44/91] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:14:57.130919 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8253b36-e3fc-480b-827f-d4e48337ed33 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:57.131224 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8253b36-e3fc-480b-827f-d4e48337ed33 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:57.131482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8253b36-e3fc-480b-827f-d4e48337ed33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:57.174837 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-f8253b36-e3fc-480b-827f-d4e48337ed33 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:57.176068 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-f8253b36-e3fc-480b-827f-d4e48337ed33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:57.208613 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d63141-d6d1-400d-ac0d-562b1e309140 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:57.246816 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c962c-16c4-49e0-a343-0916630fe6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:57.247377 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c962c-16c4-49e0-a343-0916630fe6be None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:57.247662 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c962c-16c4-49e0-a343-0916630fe6be None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:57.247942 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c962c-16c4-49e0-a343-0916630fe6be None admin] PATH_INFO: `/v3/users/8715d21c18b14387953f25f094b18c6d` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:57.249134 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c962c-16c4-49e0-a343-0916630fe6be None admin] RBAC: Authorizing `identity:get_user(user_id=8715d21c18b14387953f25f094b18c6d)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:57.250863 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c962c-16c4-49e0-a343-0916630fe6be None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:57.252467 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 48/92] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:57 2022] GET /identity/v3/users/8715d21c18b14387953f25f094b18c6d => generated 247 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:57.253733 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1965e-eff7-431a-9209-1cc6945ccebc None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:57.254022 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1965e-eff7-431a-9209-1cc6945ccebc None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:57.254299 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1965e-eff7-431a-9209-1cc6945ccebc None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:57.255530 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 49/93] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:57.256060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 45/94] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:14:57.260697 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af909a-5b83-4333-bf65-95d8c09c5899 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:57.260697 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af909a-5b83-4333-bf65-95d8c09c5899 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:57.260697 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af909a-5b83-4333-bf65-95d8c09c5899 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:57.260697 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 50/95] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:57.262204 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8253b36-e3fc-480b-827f-d4e48337ed33 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:57.263045 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddb458-e1b4-4578-b868-860f4ad88d52 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:57.263332 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddb458-e1b4-4578-b868-860f4ad88d52 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:57.263609 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddb458-e1b4-4578-b868-860f4ad88d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:57.284179 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-4cddb458-e1b4-4578-b868-860f4ad88d52 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:57.285495 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-4cddb458-e1b4-4578-b868-860f4ad88d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:57.415987 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 51/96] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:57.417569 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd67dbe-570e-4439-9a2d-7d66bb000fa5 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:57.418195 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd67dbe-570e-4439-9a2d-7d66bb000fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:57.418195 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd67dbe-570e-4439-9a2d-7d66bb000fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:57.426411 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:57.450685 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-8fd67dbe-570e-4439-9a2d-7d66bb000fa5 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:57.450685 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-8fd67dbe-570e-4439-9a2d-7d66bb000fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:57.509388 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8253b36-e3fc-480b-827f-d4e48337ed33 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:57.519666 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:57.615158 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a877b-4899-409d-b8f6-b51cafa00a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:57.615786 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a877b-4899-409d-b8f6-b51cafa00a2f None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:57.616058 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a877b-4899-409d-b8f6-b51cafa00a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:57.618411 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a877b-4899-409d-b8f6-b51cafa00a2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:57.633740 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 52/97] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:57.634676 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd67dbe-570e-4439-9a2d-7d66bb000fa5 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:57.773911 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84a877b-4899-409d-b8f6-b51cafa00a2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:57.775801 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84a877b-4899-409d-b8f6-b51cafa00a2f None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:57.784646 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd67dbe-570e-4439-9a2d-7d66bb000fa5 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:57.830129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22779a20-d244-460b-9b02-2fc60bf76eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:57.830807 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22779a20-d244-460b-9b02-2fc60bf76eaa None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:57.831087 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22779a20-d244-460b-9b02-2fc60bf76eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:57.831353 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22779a20-d244-460b-9b02-2fc60bf76eaa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:57.833318 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:57.843433 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22779a20-d244-460b-9b02-2fc60bf76eaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:57.848277 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-22779a20-d244-460b-9b02-2fc60bf76eaa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:14:57.848277 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:57.848277 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:57.848277 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:57.848277 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:57.848277 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:57.848277 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:57.848277 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:57.848277 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:57.848277 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:57.848277 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:57.848277 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:57.848277 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:57.848277 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:57.848277 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:57.848277 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:57.849239 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:57.849239 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:57.849239 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:57.849239 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:57.849239 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:57.849239 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:57.849239 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:57.849239 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:57.849239 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:57.849239 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:57.849239 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:57.849239 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:57.849239 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:57.849239 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:57.849239 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:57.849239 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:57.849994 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:57.849994 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:57.849994 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:57.849994 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:57.849994 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:57.849994 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:57.849994 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:57.849994 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:57.849994 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:57.849994 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:57.849994 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:14:57.849994 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:14:57.852771 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 53/98] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Nov 2 07:14:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:57.858363 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22779a20-d244-460b-9b02-2fc60bf76eaa None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:57.946698 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 46/99] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 07:14:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 682 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:14:57.946698 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a877b-4899-409d-b8f6-b51cafa00a2f None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:57.946943 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:57.965238 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22779a20-d244-460b-9b02-2fc60bf76eaa None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:57.998726 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5658af3-e7ca-4ae7-b3e4-2727faa42507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:57.999359 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5658af3-e7ca-4ae7-b3e4-2727faa42507 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:57.999640 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5658af3-e7ca-4ae7-b3e4-2727faa42507 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:57.999914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5658af3-e7ca-4ae7-b3e4-2727faa42507 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:58.010615 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5658af3-e7ca-4ae7-b3e4-2727faa42507 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:58.015360 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-b5658af3-e7ca-4ae7-b3e4-2727faa42507 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:14:58.015360 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:58.015360 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:58.015360 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:58.015360 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:58.015360 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:58.015360 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:58.015360 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:58.015360 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:58.015360 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:58.015360 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:58.015360 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:58.015360 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:58.015360 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:58.015360 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:58.015360 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:58.016444 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:58.016444 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:58.016444 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:58.016444 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:58.016444 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:58.016444 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:58.016444 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:58.016444 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:58.016444 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:58.016444 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:58.016444 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:58.016444 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:58.016444 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:58.016444 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:58.016444 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:58.016444 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:58.017179 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:58.017179 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:58.017179 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:58.017179 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:58.017179 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:58.017179 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:58.017179 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:58.017179 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:58.017179 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:58.017179 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:58.017179 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:14:58.017179 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:14:58.017179 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 54/100] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:58.017950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8125167-2c8a-44ac-89dc-798b58a199ae None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:58.017950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8125167-2c8a-44ac-89dc-798b58a199ae None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:58.018126 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8125167-2c8a-44ac-89dc-798b58a199ae None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:58.019478 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 55/101] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:14:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:58.020510 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb6b1f0-5652-4d43-8d86-aaf4fe0bdc64 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:58.020786 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb6b1f0-5652-4d43-8d86-aaf4fe0bdc64 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:58.021047 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb6b1f0-5652-4d43-8d86-aaf4fe0bdc64 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:58.026812 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 56/102] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:14:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:14:58.026812 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb6b1f0-5652-4d43-8d86-aaf4fe0bdc64 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:58.056482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a877b-4899-409d-b8f6-b51cafa00a2f None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:58.079071 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:58.091223 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80b640-de71-4142-98fc-aab05a9d401c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:58.092337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80b640-de71-4142-98fc-aab05a9d401c None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:58.092622 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80b640-de71-4142-98fc-aab05a9d401c None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:58.092899 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80b640-de71-4142-98fc-aab05a9d401c None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:58.093567 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab80b640-de71-4142-98fc-aab05a9d401c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:58.093871 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab80b640-de71-4142-98fc-aab05a9d401c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:58.095326 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab80b640-de71-4142-98fc-aab05a9d401c None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:58.103536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 47/103] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:14:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:58.103536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80b640-de71-4142-98fc-aab05a9d401c None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:58.131475 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb6b1f0-5652-4d43-8d86-aaf4fe0bdc64 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:58.164436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:58.165698 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6504a2-c501-4bb1-a8b1-e9399ce5badb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:58.166410 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6504a2-c501-4bb1-a8b1-e9399ce5badb None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:58.166801 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6504a2-c501-4bb1-a8b1-e9399ce5badb None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:58.167123 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6504a2-c501-4bb1-a8b1-e9399ce5badb None admin] PATH_INFO: `/v3/projects/03fe633d52c14c2aa697f9116b41facc` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:58.168315 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6504a2-c501-4bb1-a8b1-e9399ce5badb None admin] RBAC: Authorizing `identity:get_project(project_id=03fe633d52c14c2aa697f9116b41facc)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:58.169799 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6504a2-c501-4bb1-a8b1-e9399ce5badb None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:58.171471 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 57/104] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 07:14:58 2022] GET /identity/v3/projects/03fe633d52c14c2aa697f9116b41facc => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:58.172324 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6504a2-c501-4bb1-a8b1-e9399ce5badb None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:58.203087 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80b640-de71-4142-98fc-aab05a9d401c None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:58.236036 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f311f3-220c-4a08-aaac-d393d106d03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:58.236663 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f311f3-220c-4a08-aaac-d393d106d03d None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:58.236936 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f311f3-220c-4a08-aaac-d393d106d03d None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:58.237204 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f311f3-220c-4a08-aaac-d393d106d03d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:58.247994 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f311f3-220c-4a08-aaac-d393d106d03d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:58.254066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-50f311f3-220c-4a08-aaac-d393d106d03d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:14:58.254066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:58.254066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:58.254066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:58.254066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:58.254066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:58.254066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:58.254066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:58.254066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:58.254066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:58.254066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:58.254066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:58.254066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:58.254066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:58.254066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:58.254066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:58.255238 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:58.255238 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:58.255238 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:58.255238 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:58.255238 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:58.255238 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:58.255238 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:58.255238 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:58.255238 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:58.255238 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:58.255238 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:58.255238 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:58.255238 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:58.255238 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:58.255238 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:58.255238 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:58.256037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:58.256037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:58.256037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:58.256037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:58.256037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:58.256037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:58.256037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:58.256037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:58.256037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:58.256037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:58.256037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:14:58.256037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:14:58.256037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 48/105] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:58.256665 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a4e41-6461-4203-b035-84efc084ad31 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:58.256665 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a4e41-6461-4203-b035-84efc084ad31 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:58.256866 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a4e41-6461-4203-b035-84efc084ad31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:58.273988 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-ec4a4e41-6461-4203-b035-84efc084ad31 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:58.275543 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-ec4a4e41-6461-4203-b035-84efc084ad31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:58.277118 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6504a2-c501-4bb1-a8b1-e9399ce5badb None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:58.299362 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:58.312336 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561a35c-9acb-4967-9758-6d2052266fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:58.312956 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561a35c-9acb-4967-9758-6d2052266fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:58.313236 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561a35c-9acb-4967-9758-6d2052266fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:58.313508 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561a35c-9acb-4967-9758-6d2052266fe4 None admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:58.314172 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561a35c-9acb-4967-9758-6d2052266fe4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:58.314480 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561a35c-9acb-4967-9758-6d2052266fe4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:58.316295 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561a35c-9acb-4967-9758-6d2052266fe4 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:58.325031 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 58/106] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 07:14:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:58.326791 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73cfa8-fc3b-4d0a-afbe-2cfe96934743 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:58.327081 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73cfa8-fc3b-4d0a-afbe-2cfe96934743 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:58.327358 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73cfa8-fc3b-4d0a-afbe-2cfe96934743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:58.341524 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 49/107] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:58.342351 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a4e41-6461-4203-b035-84efc084ad31 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:58.355913 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-ca73cfa8-fc3b-4d0a-afbe-2cfe96934743 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:14:58.357489 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-ca73cfa8-fc3b-4d0a-afbe-2cfe96934743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:14:58.400559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:58.426081 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 59/108] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:14:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:14:58.426841 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73cfa8-fc3b-4d0a-afbe-2cfe96934743 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:58.460296 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a4e41-6461-4203-b035-84efc084ad31 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:58.501191 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51e87f-a322-4afc-82b2-76dde92fd0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:58.502260 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51e87f-a322-4afc-82b2-76dde92fd0c2 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:58.502875 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51e87f-a322-4afc-82b2-76dde92fd0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:58.503343 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51e87f-a322-4afc-82b2-76dde92fd0c2 None admin] PATH_INFO: `/v3/users/dc54984fb77c49da93c625769084ee80` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:58.504666 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f51e87f-a322-4afc-82b2-76dde92fd0c2 None admin] RBAC: Authorizing `identity:get_user(user_id=dc54984fb77c49da93c625769084ee80)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:58.506336 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f51e87f-a322-4afc-82b2-76dde92fd0c2 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:58.508786 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 50/109] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:58 2022] GET /identity/v3/users/dc54984fb77c49da93c625769084ee80 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:58.509889 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51e87f-a322-4afc-82b2-76dde92fd0c2 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:58.541045 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73cfa8-fc3b-4d0a-afbe-2cfe96934743 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:58.556236 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:58.577676 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81cea1-22e5-4f2e-81dd-78e0fdb02f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:58.578725 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81cea1-22e5-4f2e-81dd-78e0fdb02f95 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:58.579271 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81cea1-22e5-4f2e-81dd-78e0fdb02f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:58.579735 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81cea1-22e5-4f2e-81dd-78e0fdb02f95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:58.580717 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd81cea1-22e5-4f2e-81dd-78e0fdb02f95 None admin] RBAC: Adding query filter params (role.id=1e35bb725eb8475ea63e2d9538ab67d5, scope.project.id=03fe633d52c14c2aa697f9116b41facc, user.id=8715d21c18b14387953f25f094b18c6d) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:58.581332 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd81cea1-22e5-4f2e-81dd-78e0fdb02f95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:58.583109 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd81cea1-22e5-4f2e-81dd-78e0fdb02f95 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:58.589486 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 60/110] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 07:14:58 2022] GET /identity/v3/role_assignments?user.id=8715d21c18b14387953f25f094b18c6d&scope.project.id=03fe633d52c14c2aa697f9116b41facc&role.id=1e35bb725eb8475ea63e2d9538ab67d5 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:58.590737 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81cea1-22e5-4f2e-81dd-78e0fdb02f95 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:58.624222 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51e87f-a322-4afc-82b2-76dde92fd0c2 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:58.651809 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:58.666394 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe3004-73aa-44a8-ad02-ee366692afa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:58.667069 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe3004-73aa-44a8-ad02-ee366692afa1 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:58.667352 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe3004-73aa-44a8-ad02-ee366692afa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:58.667625 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe3004-73aa-44a8-ad02-ee366692afa1 None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:58.668332 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe3004-73aa-44a8-ad02-ee366692afa1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:58.668625 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe3004-73aa-44a8-ad02-ee366692afa1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:58.670069 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe3004-73aa-44a8-ad02-ee366692afa1 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:58.679856 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 51/111] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 07:14:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:58.680909 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe3004-73aa-44a8-ad02-ee366692afa1 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:58.739830 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81cea1-22e5-4f2e-81dd-78e0fdb02f95 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:58.779930 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c6ca7-d01c-427d-be5c-4db65264fcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:58.780583 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c6ca7-d01c-427d-be5c-4db65264fcb4 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:58.780872 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c6ca7-d01c-427d-be5c-4db65264fcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:58.781139 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c6ca7-d01c-427d-be5c-4db65264fcb4 None admin] PATH_INFO: `/v3/users/dc54984fb77c49da93c625769084ee80` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:58.782350 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0c6ca7-d01c-427d-be5c-4db65264fcb4 None admin] RBAC: Authorizing `identity:get_user(user_id=dc54984fb77c49da93c625769084ee80)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:58.783904 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0c6ca7-d01c-427d-be5c-4db65264fcb4 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:58.785545 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 61/112] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:58 2022] GET /identity/v3/users/dc54984fb77c49da93c625769084ee80 => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:58.790432 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c6ca7-d01c-427d-be5c-4db65264fcb4 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:58.804077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe3004-73aa-44a8-ad02-ee366692afa1 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:58.844253 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008cb8fb-94a4-49d8-9c37-352f401b8f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:58.844876 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008cb8fb-94a4-49d8-9c37-352f401b8f56 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:58.845159 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008cb8fb-94a4-49d8-9c37-352f401b8f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:58.845435 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008cb8fb-94a4-49d8-9c37-352f401b8f56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:58.850603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:58.902403 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:58.909468 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c6ca7-d01c-427d-be5c-4db65264fcb4 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:58.919008 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008cb8fb-94a4-49d8-9c37-352f401b8f56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:58.920522 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008cb8fb-94a4-49d8-9c37-352f401b8f56 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:58.942283 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec696a-02ab-4bd4-b296-b38a4f4e7927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:58.942958 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec696a-02ab-4bd4-b296-b38a4f4e7927 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:58.943242 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec696a-02ab-4bd4-b296-b38a4f4e7927 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:58.943525 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec696a-02ab-4bd4-b296-b38a4f4e7927 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:58.953425 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ec696a-02ab-4bd4-b296-b38a4f4e7927 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:14:58.958627 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-a7ec696a-02ab-4bd4-b296-b38a4f4e7927 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:14:58.958627 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:14:58.958627 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:14:58.958627 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:14:58.958627 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:14:58.958627 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:14:58.958627 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:14:58.958627 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:14:58.958627 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:14:58.958627 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:14:58.958627 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:14:58.958627 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:14:58.958627 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:14:58.958627 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:14:58.958627 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:14:58.958627 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:14:58.959536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:14:58.959536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:14:58.959536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:14:58.959536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:14:58.959536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:14:58.959536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:14:58.959536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:14:58.959536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:14:58.959536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:14:58.959536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:14:58.959536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:14:58.959536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:14:58.959536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:14:58.959536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:14:58.959536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:14:58.959536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:14:58.960823 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:14:58.960823 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:14:58.960823 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:14:58.960823 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:14:58.960823 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:14:58.960823 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:14:58.960823 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:14:58.960823 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:14:58.960823 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:14:58.960823 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:14:58.960823 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:14:58.960823 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:14:58.960823 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 62/113] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:14:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:14:58.960823 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec696a-02ab-4bd4-b296-b38a4f4e7927 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:59.023253 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 52/114] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 07:14:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:14:59.024106 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008cb8fb-94a4-49d8-9c37-352f401b8f56 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:59.062721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec696a-02ab-4bd4-b296-b38a4f4e7927 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:59.069815 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:59.097043 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367917e-45bb-404e-8e23-6fe56aa4a191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:59.097679 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367917e-45bb-404e-8e23-6fe56aa4a191 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:59.097938 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367917e-45bb-404e-8e23-6fe56aa4a191 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:59.098209 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367917e-45bb-404e-8e23-6fe56aa4a191 None admin] PATH_INFO: `/v3/users/dc54984fb77c49da93c625769084ee80` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:59.099357 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367917e-45bb-404e-8e23-6fe56aa4a191 None admin] RBAC: Authorizing `identity:get_user(user_id=dc54984fb77c49da93c625769084ee80)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:59.100823 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367917e-45bb-404e-8e23-6fe56aa4a191 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:59.102490 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 63/115] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:58 2022] GET /identity/v3/users/dc54984fb77c49da93c625769084ee80 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:59.103277 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367917e-45bb-404e-8e23-6fe56aa4a191 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:59.109603 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:59.122107 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008cb8fb-94a4-49d8-9c37-352f401b8f56 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:59.155766 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4027a7cd-87f0-4c62-a513-a9b0f67b2d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:59.156392 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027a7cd-87f0-4c62-a513-a9b0f67b2d3e None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:59.156672 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027a7cd-87f0-4c62-a513-a9b0f67b2d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:59.156947 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027a7cd-87f0-4c62-a513-a9b0f67b2d3e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:59.158041 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4027a7cd-87f0-4c62-a513-a9b0f67b2d3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:59.159613 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4027a7cd-87f0-4c62-a513-a9b0f67b2d3e None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:59.161187 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 53/116] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:14:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:59.161955 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4027a7cd-87f0-4c62-a513-a9b0f67b2d3e None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:59.204351 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367917e-45bb-404e-8e23-6fe56aa4a191 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:59.239042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7829fe10-8e9a-48a2-8c75-9bc8fec17794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:59.239658 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829fe10-8e9a-48a2-8c75-9bc8fec17794 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:59.239931 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829fe10-8e9a-48a2-8c75-9bc8fec17794 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:59.240208 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829fe10-8e9a-48a2-8c75-9bc8fec17794 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:59.261914 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4027a7cd-87f0-4c62-a513-a9b0f67b2d3e None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:59.283327 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:59.296114 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3ac25-dcbf-45e1-b747-0d16e7ec0442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:59.296728 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3ac25-dcbf-45e1-b747-0d16e7ec0442 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:59.297004 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3ac25-dcbf-45e1-b747-0d16e7ec0442 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:59.297279 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3ac25-dcbf-45e1-b747-0d16e7ec0442 None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:59.297940 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a3ac25-dcbf-45e1-b747-0d16e7ec0442 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:59.298243 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a3ac25-dcbf-45e1-b747-0d16e7ec0442 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:59.299667 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a3ac25-dcbf-45e1-b747-0d16e7ec0442 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:59.307731 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 54/117] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 07:14:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:59.308534 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3ac25-dcbf-45e1-b747-0d16e7ec0442 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:59.310077 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7829fe10-8e9a-48a2-8c75-9bc8fec17794 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:59.312617 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7829fe10-8e9a-48a2-8c75-9bc8fec17794 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:59.324286 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:59.406375 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3ac25-dcbf-45e1-b747-0d16e7ec0442 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:59.410190 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 64/118] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 07:14:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:14:59.411067 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7829fe10-8e9a-48a2-8c75-9bc8fec17794 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:59.439815 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f36ea-e147-45d6-9575-ff7ba9558725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:59.440440 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f36ea-e147-45d6-9575-ff7ba9558725 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:59.440720 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f36ea-e147-45d6-9575-ff7ba9558725 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:59.440987 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f36ea-e147-45d6-9575-ff7ba9558725 None admin] PATH_INFO: `/v3/projects/03fe633d52c14c2aa697f9116b41facc` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:59.442092 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063f36ea-e147-45d6-9575-ff7ba9558725 None admin] RBAC: Authorizing `identity:get_project(project_id=03fe633d52c14c2aa697f9116b41facc)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:59.443575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063f36ea-e147-45d6-9575-ff7ba9558725 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:59.445226 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 55/119] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 07:14:59 2022] GET /identity/v3/projects/03fe633d52c14c2aa697f9116b41facc => generated 298 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:59.446071 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f36ea-e147-45d6-9575-ff7ba9558725 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:59.487005 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:59.511384 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7829fe10-8e9a-48a2-8c75-9bc8fec17794 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:59.527232 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:59.545189 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c8a85-2bcf-446b-954c-e334a3debbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:59.545605 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f36ea-e147-45d6-9575-ff7ba9558725 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:59.545810 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c8a85-2bcf-446b-954c-e334a3debbc7 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:59.546088 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c8a85-2bcf-446b-954c-e334a3debbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:59.546359 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c8a85-2bcf-446b-954c-e334a3debbc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:59.578740 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d82fc-610e-404e-a80e-b9169b3ffe77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:59.579294 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d82fc-610e-404e-a80e-b9169b3ffe77 None admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:59.579579 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d82fc-610e-404e-a80e-b9169b3ffe77 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:59.579872 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d82fc-610e-404e-a80e-b9169b3ffe77 None admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:59.580508 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696d82fc-610e-404e-a80e-b9169b3ffe77 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:59.581965 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696d82fc-610e-404e-a80e-b9169b3ffe77 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:59.611222 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 56/120] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Nov 2 07:14:59 2022] POST /identity/v3/projects => generated 302 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:14:59.612255 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d82fc-610e-404e-a80e-b9169b3ffe77 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:59.614562 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c8a85-2bcf-446b-954c-e334a3debbc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:59.616112 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c8a85-2bcf-446b-954c-e334a3debbc7 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:59.717278 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:59.734498 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d82fc-610e-404e-a80e-b9169b3ffe77 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:59.749915 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 65/121] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 07:14:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:14:59.751582 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c8a85-2bcf-446b-954c-e334a3debbc7 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:59.775163 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4bdbdb-0e3b-4463-8192-2c1eb0bc1186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:59.776178 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bdbdb-0e3b-4463-8192-2c1eb0bc1186 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:59.776460 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bdbdb-0e3b-4463-8192-2c1eb0bc1186 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:59.776734 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bdbdb-0e3b-4463-8192-2c1eb0bc1186 None admin] PATH_INFO: `/v3/users/662c18ae9452498cbbf56e6230b00403` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:59.779587 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:14:59.793826 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4bdbdb-0e3b-4463-8192-2c1eb0bc1186 None admin] RBAC: Authorizing `identity:get_user(user_id=662c18ae9452498cbbf56e6230b00403)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:59.795486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4bdbdb-0e3b-4463-8192-2c1eb0bc1186 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:59.797386 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 57/122] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:14:59 2022] GET /identity/v3/users/662c18ae9452498cbbf56e6230b00403 => generated 289 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:59.798171 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4bdbdb-0e3b-4463-8192-2c1eb0bc1186 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:59.873945 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c8a85-2bcf-446b-954c-e334a3debbc7 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:59.906313 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4bdbdb-0e3b-4463-8192-2c1eb0bc1186 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:14:59.908567 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4cfc15-305d-48a9-80c0-49d990760138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:59.909234 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4cfc15-305d-48a9-80c0-49d990760138 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:59.909508 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4cfc15-305d-48a9-80c0-49d990760138 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:59.909780 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4cfc15-305d-48a9-80c0-49d990760138 None admin] PATH_INFO: `/v3/projects/60c61605c0a6402485c7e69d18304dba` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:59.910920 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4cfc15-305d-48a9-80c0-49d990760138 None admin] RBAC: Authorizing `identity:get_project(project_id=60c61605c0a6402485c7e69d18304dba)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:59.912386 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4cfc15-305d-48a9-80c0-49d990760138 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:59.914092 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 66/123] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 07:14:59 2022] GET /identity/v3/projects/60c61605c0a6402485c7e69d18304dba => generated 312 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:59.914905 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4cfc15-305d-48a9-80c0-49d990760138 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:59.948102 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b1b50-007a-4c44-be34-52bd58235534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:14:59.948724 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b1b50-007a-4c44-be34-52bd58235534 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:14:59.949312 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b1b50-007a-4c44-be34-52bd58235534 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:14:59.949625 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b1b50-007a-4c44-be34-52bd58235534 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:14:59.950313 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b1b50-007a-4c44-be34-52bd58235534 None admin] RBAC: Adding query filter params (role.id=39eab44f330440ebbefe39e538462165, scope.project.id=03fe633d52c14c2aa697f9116b41facc, user.id=dc54984fb77c49da93c625769084ee80) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:14:59.950617 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b1b50-007a-4c44-be34-52bd58235534 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:14:59.952087 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b1b50-007a-4c44-be34-52bd58235534 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:14:59.966309 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 58/124] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 07:14:59 2022] GET /identity/v3/role_assignments?user.id=dc54984fb77c49da93c625769084ee80&scope.project.id=03fe633d52c14c2aa697f9116b41facc&role.id=39eab44f330440ebbefe39e538462165 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:14:59.966309 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b1b50-007a-4c44-be34-52bd58235534 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:14:59.973269 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:00.040045 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4cfc15-305d-48a9-80c0-49d990760138 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:00.047088 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:00.082607 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925b3bd-b00a-42a4-ac38-82fe6788e736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:00.083726 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925b3bd-b00a-42a4-ac38-82fe6788e736 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:00.084148 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925b3bd-b00a-42a4-ac38-82fe6788e736 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:00.084603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925b3bd-b00a-42a4-ac38-82fe6788e736 None admin] PATH_INFO: `/v3/projects/03fe633d52c14c2aa697f9116b41facc` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:00.086146 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3925b3bd-b00a-42a4-ac38-82fe6788e736 None admin] RBAC: Authorizing `identity:get_project(project_id=03fe633d52c14c2aa697f9116b41facc)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:00.087962 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3925b3bd-b00a-42a4-ac38-82fe6788e736 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:00.094923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 67/125] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 07:14:59 2022] GET /identity/v3/projects/03fe633d52c14c2aa697f9116b41facc => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:00.096300 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925b3bd-b00a-42a4-ac38-82fe6788e736 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:00.104394 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b1b50-007a-4c44-be34-52bd58235534 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:00.152894 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e49c0d-fe5a-4c01-963a-79beab713881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:00.153519 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e49c0d-fe5a-4c01-963a-79beab713881 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:00.153716 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e49c0d-fe5a-4c01-963a-79beab713881 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:00.153985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e49c0d-fe5a-4c01-963a-79beab713881 None admin] PATH_INFO: `/v3/projects/03fe633d52c14c2aa697f9116b41facc` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:00.155889 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e49c0d-fe5a-4c01-963a-79beab713881 None admin] RBAC: Authorizing `identity:get_project(project_id=03fe633d52c14c2aa697f9116b41facc)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:00.157547 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e49c0d-fe5a-4c01-963a-79beab713881 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:00.159649 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 59/126] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 07:14:59 2022] GET /identity/v3/projects/03fe633d52c14c2aa697f9116b41facc => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:00.160311 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e49c0d-fe5a-4c01-963a-79beab713881 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:00.224803 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925b3bd-b00a-42a4-ac38-82fe6788e736 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:00.236740 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:00.269439 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0601743-635f-4d54-8307-2236b8fde16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:00.270067 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0601743-635f-4d54-8307-2236b8fde16d None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:00.270347 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0601743-635f-4d54-8307-2236b8fde16d None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:00.270623 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0601743-635f-4d54-8307-2236b8fde16d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:00.271396 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0601743-635f-4d54-8307-2236b8fde16d None admin] RBAC: Adding query filter params (role.id=15930dae1c0b464f98680edb03b7f02c, scope.project.id=60c61605c0a6402485c7e69d18304dba, user.id=dc54984fb77c49da93c625769084ee80) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:00.271707 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0601743-635f-4d54-8307-2236b8fde16d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:00.273162 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0601743-635f-4d54-8307-2236b8fde16d None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:00.284981 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e49c0d-fe5a-4c01-963a-79beab713881 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:00.286993 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 68/127] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 07:15:00 2022] GET /identity/v3/role_assignments?user.id=dc54984fb77c49da93c625769084ee80&scope.project.id=60c61605c0a6402485c7e69d18304dba&role.id=15930dae1c0b464f98680edb03b7f02c => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:00.287797 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0601743-635f-4d54-8307-2236b8fde16d None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:00.317292 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:00.333749 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c70946-bad7-4f60-b073-afdc7908a5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:00.334547 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c70946-bad7-4f60-b073-afdc7908a5f2 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:00.334919 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c70946-bad7-4f60-b073-afdc7908a5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:00.335259 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c70946-bad7-4f60-b073-afdc7908a5f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:00.336809 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c70946-bad7-4f60-b073-afdc7908a5f2 None admin] RBAC: Adding query filter params (role.id=15930dae1c0b464f98680edb03b7f02c, scope.project.id=03fe633d52c14c2aa697f9116b41facc, user.id=dc54984fb77c49da93c625769084ee80) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:00.337198 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c70946-bad7-4f60-b073-afdc7908a5f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:00.339960 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c70946-bad7-4f60-b073-afdc7908a5f2 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:00.349104 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 60/128] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 07:15:00 2022] GET /identity/v3/role_assignments?user.id=dc54984fb77c49da93c625769084ee80&scope.project.id=03fe633d52c14c2aa697f9116b41facc&role.id=15930dae1c0b464f98680edb03b7f02c => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:00.428869 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0601743-635f-4d54-8307-2236b8fde16d None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:00.471029 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d47ec-97b2-462c-9ab1-b91abcdfe95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:00.471774 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d47ec-97b2-462c-9ab1-b91abcdfe95a None admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:00.472053 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d47ec-97b2-462c-9ab1-b91abcdfe95a None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:00.472335 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d47ec-97b2-462c-9ab1-b91abcdfe95a None admin] PATH_INFO: `/v3/projects/03fe633d52c14c2aa697f9116b41facc/users/662c18ae9452498cbbf56e6230b00403/roles/0ca4aff56e484fa99225a67bae03722f` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:00.474488 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d47ec-97b2-462c-9ab1-b91abcdfe95a None admin] RBAC: Authorizing `identity:create_grant(project_id=03fe633d52c14c2aa697f9116b41facc, user_id=662c18ae9452498cbbf56e6230b00403, role_id=0ca4aff56e484fa99225a67bae03722f)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:00.476069 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d47ec-97b2-462c-9ab1-b91abcdfe95a None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:00.497341 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 69/129] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 07:15:00 2022] PUT /identity/v3/projects/03fe633d52c14c2aa697f9116b41facc/users/662c18ae9452498cbbf56e6230b00403/roles/0ca4aff56e484fa99225a67bae03722f => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:01.825488 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d22ef-a777-4a51-a3eb-adc9e6e60974 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:01.826007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d22ef-a777-4a51-a3eb-adc9e6e60974 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:01.826007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d22ef-a777-4a51-a3eb-adc9e6e60974 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:01.827574 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 61/130] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:01.833062 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324505a-a0ca-45c9-b370-046700d0e565 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:01.833491 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324505a-a0ca-45c9-b370-046700d0e565 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:01.833569 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324505a-a0ca-45c9-b370-046700d0e565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:01.852165 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-e324505a-a0ca-45c9-b370-046700d0e565 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:01.853355 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-e324505a-a0ca-45c9-b370-046700d0e565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:01.881880 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:01.926120 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 70/131] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:02.193415 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d22ef-a777-4a51-a3eb-adc9e6e60974 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:02.302358 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d22ef-a777-4a51-a3eb-adc9e6e60974 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:02.337158 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fcffe8-7a46-444d-8182-613d81ca8ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:02.337781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fcffe8-7a46-444d-8182-613d81ca8ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:02.338060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fcffe8-7a46-444d-8182-613d81ca8ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:02.338334 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fcffe8-7a46-444d-8182-613d81ca8ff6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:02.339502 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fcffe8-7a46-444d-8182-613d81ca8ff6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:02.340974 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fcffe8-7a46-444d-8182-613d81ca8ff6 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:02.342539 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 62/132] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:02.348245 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324505a-a0ca-45c9-b370-046700d0e565 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:02.451292 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324505a-a0ca-45c9-b370-046700d0e565 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:02.486147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a606377-840d-4d60-8c59-a723165d285d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:02.486807 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a606377-840d-4d60-8c59-a723165d285d None admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:02.487091 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a606377-840d-4d60-8c59-a723165d285d None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:02.487365 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a606377-840d-4d60-8c59-a723165d285d None admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:02.488013 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a606377-840d-4d60-8c59-a723165d285d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:02.489470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a606377-840d-4d60-8c59-a723165d285d None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:02.513747 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 71/133] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 07:15:02 2022] POST /identity/v3/users => generated 283 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:15:02.641603 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56355bdd-0b3a-47bd-b407-def905cec4b4 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:02.641850 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56355bdd-0b3a-47bd-b407-def905cec4b4 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:02.642156 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56355bdd-0b3a-47bd-b407-def905cec4b4 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:02.643579 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 63/134] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:02.650427 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8dd3c7-5ceb-4451-8932-eb91e6e4a968 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:02.650741 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8dd3c7-5ceb-4451-8932-eb91e6e4a968 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:02.651371 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8dd3c7-5ceb-4451-8932-eb91e6e4a968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:02.671795 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-6e8dd3c7-5ceb-4451-8932-eb91e6e4a968 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:02.673016 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-6e8dd3c7-5ceb-4451-8932-eb91e6e4a968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:02.689157 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:02.750315 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 72/135] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:03.024000 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56355bdd-0b3a-47bd-b407-def905cec4b4 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:03.082809 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:03.130048 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56355bdd-0b3a-47bd-b407-def905cec4b4 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:03.165021 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8c97a-a60b-418f-92d1-ac104ec3027c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:03.165691 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8c97a-a60b-418f-92d1-ac104ec3027c None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:03.166005 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8c97a-a60b-418f-92d1-ac104ec3027c None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:03.166282 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8c97a-a60b-418f-92d1-ac104ec3027c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:03.176879 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e8c97a-a60b-418f-92d1-ac104ec3027c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:03.182230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-c7e8c97a-a60b-418f-92d1-ac104ec3027c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:15:03.182230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:03.182230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:03.182230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:03.182230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:03.182230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:03.182230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:03.182230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:03.182230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:03.182230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:03.182230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:03.182230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:03.182230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:03.182230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:03.182230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:03.182230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:03.183099 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:03.183099 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:03.183099 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:03.183099 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:03.183099 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:03.183099 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:03.183099 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:03.183099 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:03.183099 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:03.183099 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:03.183099 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:03.183099 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:03.183099 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:03.183099 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:03.183099 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:03.183099 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:03.183841 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:03.183841 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:03.183841 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:03.183841 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:03.183841 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:03.183841 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:03.183841 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:03.183841 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:03.183841 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:03.183841 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:03.183841 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:15:03.183841 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:03.183841 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 64/136] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:03.190683 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8dd3c7-5ceb-4451-8932-eb91e6e4a968 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:03.294442 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8dd3c7-5ceb-4451-8932-eb91e6e4a968 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:03.329398 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69149b9b-4916-4ef6-af87-51eae1e668be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:03.330011 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69149b9b-4916-4ef6-af87-51eae1e668be None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:03.330284 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69149b9b-4916-4ef6-af87-51eae1e668be None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:03.330622 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69149b9b-4916-4ef6-af87-51eae1e668be None admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:03.331384 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69149b9b-4916-4ef6-af87-51eae1e668be None admin] RBAC: Adding query filter params (name=reader) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:03.331671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69149b9b-4916-4ef6-af87-51eae1e668be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:03.333118 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69149b9b-4916-4ef6-af87-51eae1e668be None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:03.342152 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 73/137] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 07:15:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:03.348033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8c97a-a60b-418f-92d1-ac104ec3027c None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:03.450122 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8c97a-a60b-418f-92d1-ac104ec3027c None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:03.466721 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:03.486105 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a090eb7-b196-4119-ab0a-98d30de1ffa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:03.486781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a090eb7-b196-4119-ab0a-98d30de1ffa2 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:03.487059 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a090eb7-b196-4119-ab0a-98d30de1ffa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:03.487330 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a090eb7-b196-4119-ab0a-98d30de1ffa2 None admin] PATH_INFO: `/v3/users/662c18ae9452498cbbf56e6230b00403` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:03.488548 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a090eb7-b196-4119-ab0a-98d30de1ffa2 None admin] RBAC: Authorizing `identity:get_user(user_id=662c18ae9452498cbbf56e6230b00403)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:03.490083 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a090eb7-b196-4119-ab0a-98d30de1ffa2 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:03.496812 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 65/138] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:03 2022] GET /identity/v3/users/662c18ae9452498cbbf56e6230b00403 => generated 289 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:03.498382 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69149b9b-4916-4ef6-af87-51eae1e668be None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:03.527053 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:03.603297 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69149b9b-4916-4ef6-af87-51eae1e668be None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:03.637437 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e8f74-2919-4665-83bf-7c76e43cf20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:03.638050 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e8f74-2919-4665-83bf-7c76e43cf20e None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:03.638368 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e8f74-2919-4665-83bf-7c76e43cf20e None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:03.638635 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e8f74-2919-4665-83bf-7c76e43cf20e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:03.712493 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e8f74-2919-4665-83bf-7c76e43cf20e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:03.714121 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e8f74-2919-4665-83bf-7c76e43cf20e None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:03.728386 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:03.813442 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 74/139] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:15:03.819296 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a090eb7-b196-4119-ab0a-98d30de1ffa2 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:03.920197 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a090eb7-b196-4119-ab0a-98d30de1ffa2 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:03.958184 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da6a63-c6a9-4300-a5e0-eb17e9801418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:03.959222 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da6a63-c6a9-4300-a5e0-eb17e9801418 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:03.959694 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da6a63-c6a9-4300-a5e0-eb17e9801418 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:03.960147 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da6a63-c6a9-4300-a5e0-eb17e9801418 None admin] PATH_INFO: `/v3/projects/03fe633d52c14c2aa697f9116b41facc` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:03.961605 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79da6a63-c6a9-4300-a5e0-eb17e9801418 None admin] RBAC: Authorizing `identity:get_project(project_id=03fe633d52c14c2aa697f9116b41facc)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:03.963479 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79da6a63-c6a9-4300-a5e0-eb17e9801418 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:03.965311 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 66/140] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 07:15:03 2022] GET /identity/v3/projects/03fe633d52c14c2aa697f9116b41facc => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:03.971481 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e8f74-2919-4665-83bf-7c76e43cf20e None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:04.075247 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e8f74-2919-4665-83bf-7c76e43cf20e None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:04.089402 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:04.110960 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f5f46-f1ed-47c6-af4d-dcb72571a5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:04.111569 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f5f46-f1ed-47c6-af4d-dcb72571a5b9 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:04.111841 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f5f46-f1ed-47c6-af4d-dcb72571a5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:04.112110 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f5f46-f1ed-47c6-af4d-dcb72571a5b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:04.112830 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f5f46-f1ed-47c6-af4d-dcb72571a5b9 None admin] RBAC: Adding query filter params (role.id=0ca4aff56e484fa99225a67bae03722f, scope.project.id=03fe633d52c14c2aa697f9116b41facc, user.id=662c18ae9452498cbbf56e6230b00403) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:04.113129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f5f46-f1ed-47c6-af4d-dcb72571a5b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:04.114566 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f5f46-f1ed-47c6-af4d-dcb72571a5b9 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:04.126792 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 75/141] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 07:15:03 2022] GET /identity/v3/role_assignments?user.id=662c18ae9452498cbbf56e6230b00403&scope.project.id=03fe633d52c14c2aa697f9116b41facc&role.id=0ca4aff56e484fa99225a67bae03722f => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:04.795615 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8e69e-683f-4437-a95b-9118012edfbb None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:04.796201 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8e69e-683f-4437-a95b-9118012edfbb None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:04.796201 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8e69e-683f-4437-a95b-9118012edfbb None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:04.797477 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 67/142] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:04.802824 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51ff08-a836-49e6-8696-4f2404de1045 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:04.803258 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51ff08-a836-49e6-8696-4f2404de1045 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:04.803334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51ff08-a836-49e6-8696-4f2404de1045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:04.823645 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-ee51ff08-a836-49e6-8696-4f2404de1045 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:04.824903 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-ee51ff08-a836-49e6-8696-4f2404de1045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:04.898176 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 76/143] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:04.942187 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7d793-aaad-4f6c-8533-7c98b57a1b70 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:04.942440 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7d793-aaad-4f6c-8533-7c98b57a1b70 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:04.942942 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7d793-aaad-4f6c-8533-7c98b57a1b70 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:04.944268 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 68/144] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:04.950152 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5196e4-4ff7-465c-90f3-ae68d9409ff0 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:04.950442 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5196e4-4ff7-465c-90f3-ae68d9409ff0 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:04.950743 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5196e4-4ff7-465c-90f3-ae68d9409ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:04.969070 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-cc5196e4-4ff7-465c-90f3-ae68d9409ff0 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:04.970251 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-cc5196e4-4ff7-465c-90f3-ae68d9409ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:05.019755 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c1dcd-b389-4aa1-9727-d7f7b957949f None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:05.020182 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c1dcd-b389-4aa1-9727-d7f7b957949f None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:05.020884 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c1dcd-b389-4aa1-9727-d7f7b957949f None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:05.022369 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 69/145] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:05.028251 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181db16-a1e1-46fb-b5e7-56e156444d38 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:05.028530 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181db16-a1e1-46fb-b5e7-56e156444d38 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:05.028800 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181db16-a1e1-46fb-b5e7-56e156444d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:05.044605 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 77/146] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:05.047946 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-5181db16-a1e1-46fb-b5e7-56e156444d38 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:05.049137 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-5181db16-a1e1-46fb-b5e7-56e156444d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:05.097929 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:05.118462 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 70/147] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:05.164890 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5196e4-4ff7-465c-90f3-ae68d9409ff0 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:05.202803 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:05.270938 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5196e4-4ff7-465c-90f3-ae68d9409ff0 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:05.307963 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e42b4-70fa-4f69-961f-b793034161c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:05.308291 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181db16-a1e1-46fb-b5e7-56e156444d38 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:05.308619 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e42b4-70fa-4f69-961f-b793034161c3 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:05.308900 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e42b4-70fa-4f69-961f-b793034161c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:05.309169 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e42b4-70fa-4f69-961f-b793034161c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:05.321017 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797e42b4-70fa-4f69-961f-b793034161c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:05.326042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-797e42b4-70fa-4f69-961f-b793034161c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:05.326042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:05.326042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:05.326042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:05.326042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:05.326042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:05.326042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:05.326042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:05.326042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:05.326042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:05.326042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:05.326042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:05.326042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:05.326042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:05.326042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:05.326042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:05.326933 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:05.326933 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:05.326933 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:05.326933 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:05.326933 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:05.326933 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:05.326933 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:05.326933 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:05.326933 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:05.326933 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:05.326933 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:05.326933 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:05.326933 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:05.326933 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:05.326933 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:05.326933 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:05.327898 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:05.327898 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:05.327898 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:05.327898 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:05.327898 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:05.327898 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:05.327898 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:05.327898 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:05.327898 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:05.327898 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:05.327898 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:05.327898 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:05.327898 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 78/148] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 07:15:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:05.332845 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e42b4-70fa-4f69-961f-b793034161c3 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:05.413818 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181db16-a1e1-46fb-b5e7-56e156444d38 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:05.428646 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:05.434965 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e42b4-70fa-4f69-961f-b793034161c3 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:05.447615 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ae746-30ac-48c5-abee-7c94817b835a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:05.448256 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ae746-30ac-48c5-abee-7c94817b835a None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:05.448532 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ae746-30ac-48c5-abee-7c94817b835a None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:05.448805 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ae746-30ac-48c5-abee-7c94817b835a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:05.450024 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ae746-30ac-48c5-abee-7c94817b835a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:05.451543 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ae746-30ac-48c5-abee-7c94817b835a None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:05.453119 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 71/149] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:05.457814 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ae746-30ac-48c5-abee-7c94817b835a None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:05.472578 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e688d-d84f-4558-a306-e07d42a4399d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:05.473191 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e688d-d84f-4558-a306-e07d42a4399d None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:05.473469 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e688d-d84f-4558-a306-e07d42a4399d None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:05.473726 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e688d-d84f-4558-a306-e07d42a4399d None admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:05.474403 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e688d-d84f-4558-a306-e07d42a4399d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:05.474726 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e688d-d84f-4558-a306-e07d42a4399d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:05.476123 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e688d-d84f-4558-a306-e07d42a4399d None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:05.484834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 79/150] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:05.484834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e688d-d84f-4558-a306-e07d42a4399d None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:05.507172 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:05.559280 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ae746-30ac-48c5-abee-7c94817b835a None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:05.585002 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e688d-d84f-4558-a306-e07d42a4399d None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:05.592845 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0ce17-c623-468b-af8a-ee8b52ab687b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:05.593484 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0ce17-c623-468b-af8a-ee8b52ab687b None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:05.593751 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0ce17-c623-468b-af8a-ee8b52ab687b None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:05.594025 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0ce17-c623-468b-af8a-ee8b52ab687b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:05.605089 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb0ce17-c623-468b-af8a-ee8b52ab687b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:05.609963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-8bb0ce17-c623-468b-af8a-ee8b52ab687b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:15:05.609963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:05.609963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:05.609963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:05.609963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:05.609963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:05.609963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:05.609963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:05.609963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:05.609963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:05.609963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:05.609963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:05.609963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:05.609963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:05.609963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:05.609963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:05.610979 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:05.610979 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:05.610979 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:05.610979 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:05.610979 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:05.610979 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:05.610979 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:05.610979 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:05.610979 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:05.610979 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:05.610979 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:05.610979 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:05.610979 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:05.610979 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:05.610979 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:05.610979 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:05.615598 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:05.615598 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:05.615598 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:05.615598 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:05.615598 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:05.615598 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:05.615598 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:05.615598 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:05.615598 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:05.615598 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:05.615598 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:15:05.615598 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:05.615598 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 72/151] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Nov 2 07:15:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:05.615598 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0ce17-c623-468b-af8a-ee8b52ab687b None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:05.629039 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c47a99d-715b-46d7-971a-47b708f85099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:05.629657 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c47a99d-715b-46d7-971a-47b708f85099 None admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:05.629934 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c47a99d-715b-46d7-971a-47b708f85099 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:05.630198 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c47a99d-715b-46d7-971a-47b708f85099 None admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:05.630869 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c47a99d-715b-46d7-971a-47b708f85099 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:05.632332 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c47a99d-715b-46d7-971a-47b708f85099 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:05.654224 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 80/152] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 07:15:05 2022] POST /identity/v3/users => generated 297 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:15:05.660069 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c47a99d-715b-46d7-971a-47b708f85099 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:05.679605 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:05.732857 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0ce17-c623-468b-af8a-ee8b52ab687b None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:05.747681 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:05.785380 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c47a99d-715b-46d7-971a-47b708f85099 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:05.786933 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f45476-2e18-4600-8c2a-911431689f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:05.787599 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f45476-2e18-4600-8c2a-911431689f77 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:05.787891 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f45476-2e18-4600-8c2a-911431689f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:05.788161 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f45476-2e18-4600-8c2a-911431689f77 None admin] PATH_INFO: `/v3/users/280642cc66c84947b6bceab16f065990` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:05.804359 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f45476-2e18-4600-8c2a-911431689f77 None admin] RBAC: Authorizing `identity:get_user(user_id=280642cc66c84947b6bceab16f065990)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:05.805868 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f45476-2e18-4600-8c2a-911431689f77 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:05.807630 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 73/153] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:05 2022] GET /identity/v3/users/280642cc66c84947b6bceab16f065990 => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:05.813250 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f45476-2e18-4600-8c2a-911431689f77 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:05.827399 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b38d128-6c33-4028-96a3-e0ca71211428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:05.828069 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38d128-6c33-4028-96a3-e0ca71211428 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:05.828337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38d128-6c33-4028-96a3-e0ca71211428 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:05.828579 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38d128-6c33-4028-96a3-e0ca71211428 None admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:05.829254 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b38d128-6c33-4028-96a3-e0ca71211428 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:05.829556 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b38d128-6c33-4028-96a3-e0ca71211428 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:05.830993 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b38d128-6c33-4028-96a3-e0ca71211428 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:05.843929 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 81/154] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:05.849455 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b38d128-6c33-4028-96a3-e0ca71211428 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:05.933791 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f45476-2e18-4600-8c2a-911431689f77 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:05.937469 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:05.964152 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b38d128-6c33-4028-96a3-e0ca71211428 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:05.974495 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daaf022-6e18-4138-844c-bd75a27eb0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:05.975543 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daaf022-6e18-4138-844c-bd75a27eb0a6 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:05.976001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daaf022-6e18-4138-844c-bd75a27eb0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:05.976428 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daaf022-6e18-4138-844c-bd75a27eb0a6 None admin] PATH_INFO: `/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:05.978613 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daaf022-6e18-4138-844c-bd75a27eb0a6 None admin] RBAC: Authorizing `identity:get_project(project_id=ef9f41b2925a4825a22ae82a7bd2e955)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:05.980351 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daaf022-6e18-4138-844c-bd75a27eb0a6 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:05.982468 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 74/155] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 07:15:05 2022] GET /identity/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:05.988223 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daaf022-6e18-4138-844c-bd75a27eb0a6 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:06.004824 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e949c-4ca4-42b0-bbdb-741685ea111a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:06.005477 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e949c-4ca4-42b0-bbdb-741685ea111a None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:06.005752 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e949c-4ca4-42b0-bbdb-741685ea111a None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:06.006022 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e949c-4ca4-42b0-bbdb-741685ea111a None admin] PATH_INFO: `/v3/users/280642cc66c84947b6bceab16f065990` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:06.007183 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e949c-4ca4-42b0-bbdb-741685ea111a None admin] RBAC: Authorizing `identity:get_user(user_id=280642cc66c84947b6bceab16f065990)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:06.009042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e949c-4ca4-42b0-bbdb-741685ea111a None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:06.010870 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 82/156] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:05 2022] GET /identity/v3/users/280642cc66c84947b6bceab16f065990 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:06.017739 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e949c-4ca4-42b0-bbdb-741685ea111a None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:06.027667 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:06.105931 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daaf022-6e18-4138-844c-bd75a27eb0a6 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:06.134818 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e949c-4ca4-42b0-bbdb-741685ea111a None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:06.152848 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd435c6f-3ddf-4f3d-95c4-b530c63f8ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:06.154213 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd435c6f-3ddf-4f3d-95c4-b530c63f8ec4 None admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:06.154734 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd435c6f-3ddf-4f3d-95c4-b530c63f8ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:06.155382 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd435c6f-3ddf-4f3d-95c4-b530c63f8ec4 None admin] PATH_INFO: `/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955/users/280642cc66c84947b6bceab16f065990/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:06.158375 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd435c6f-3ddf-4f3d-95c4-b530c63f8ec4 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef9f41b2925a4825a22ae82a7bd2e955, user_id=280642cc66c84947b6bceab16f065990, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:06.160612 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd435c6f-3ddf-4f3d-95c4-b530c63f8ec4 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:06.171612 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574339fe-a6a1-4cb8-8b42-6f0fa3ebe0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:06.172493 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574339fe-a6a1-4cb8-8b42-6f0fa3ebe0cc None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:06.172922 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574339fe-a6a1-4cb8-8b42-6f0fa3ebe0cc None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:06.173334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574339fe-a6a1-4cb8-8b42-6f0fa3ebe0cc None admin] PATH_INFO: `/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:06.174618 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574339fe-a6a1-4cb8-8b42-6f0fa3ebe0cc None admin] RBAC: Authorizing `identity:get_project(project_id=ef9f41b2925a4825a22ae82a7bd2e955)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:06.176235 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574339fe-a6a1-4cb8-8b42-6f0fa3ebe0cc None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:06.178061 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 83/157] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 07:15:06 2022] GET /identity/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:06.178980 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 75/158] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 07:15:05 2022] PUT /identity/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955/users/280642cc66c84947b6bceab16f065990/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:15:06.184522 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574339fe-a6a1-4cb8-8b42-6f0fa3ebe0cc None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:06.203214 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:06.304091 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574339fe-a6a1-4cb8-8b42-6f0fa3ebe0cc None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:06.348335 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae6b406-e7c5-4654-95e4-13af45dc1a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:06.349034 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6b406-e7c5-4654-95e4-13af45dc1a37 None admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:06.349314 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6b406-e7c5-4654-95e4-13af45dc1a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:06.349587 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6b406-e7c5-4654-95e4-13af45dc1a37 None admin] PATH_INFO: `/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955/users/280642cc66c84947b6bceab16f065990/roles/39eab44f330440ebbefe39e538462165` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:06.352220 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae6b406-e7c5-4654-95e4-13af45dc1a37 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef9f41b2925a4825a22ae82a7bd2e955, user_id=280642cc66c84947b6bceab16f065990, role_id=39eab44f330440ebbefe39e538462165)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:06.354063 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae6b406-e7c5-4654-95e4-13af45dc1a37 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:06.370972 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 84/159] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 07:15:06 2022] PUT /identity/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955/users/280642cc66c84947b6bceab16f065990/roles/39eab44f330440ebbefe39e538462165 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:07.956958 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1528740-b2a1-4546-bfba-517bd09d8516 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:07.957477 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1528740-b2a1-4546-bfba-517bd09d8516 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:07.957477 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1528740-b2a1-4546-bfba-517bd09d8516 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:07.959007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 76/160] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:07.964331 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a934e-901b-4009-9ee5-d2b899683d12 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:07.964760 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a934e-901b-4009-9ee5-d2b899683d12 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:07.964950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a934e-901b-4009-9ee5-d2b899683d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:07.983645 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-637a934e-901b-4009-9ee5-d2b899683d12 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:07.984847 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-637a934e-901b-4009-9ee5-d2b899683d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:08.059239 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 85/161] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:08.139258 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08633c7-4949-45bb-b490-ddf13f21d39d None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:08.139570 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08633c7-4949-45bb-b490-ddf13f21d39d None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:08.139805 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08633c7-4949-45bb-b490-ddf13f21d39d None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:08.141176 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 77/162] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:08.146720 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed1997-be5d-4fb0-aa72-4b86da65d8cf None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:08.147250 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed1997-be5d-4fb0-aa72-4b86da65d8cf None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:08.147734 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed1997-be5d-4fb0-aa72-4b86da65d8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:08.154837 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:08.169053 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-54ed1997-be5d-4fb0-aa72-4b86da65d8cf None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:08.170627 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-54ed1997-be5d-4fb0-aa72-4b86da65d8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:08.240770 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 86/163] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:08.343243 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08633c7-4949-45bb-b490-ddf13f21d39d None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:08.394602 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4dca8-635e-4129-88a0-3d37b7d4dd8b None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:08.394971 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4dca8-635e-4129-88a0-3d37b7d4dd8b None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:08.395193 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4dca8-635e-4129-88a0-3d37b7d4dd8b None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:08.396564 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 87/164] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:15:08.402577 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e06d0-0938-4ffa-9ae8-a280c0b8a86c None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:08.403166 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e06d0-0938-4ffa-9ae8-a280c0b8a86c None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:08.403633 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e06d0-0938-4ffa-9ae8-a280c0b8a86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:08.422965 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-d00e06d0-0938-4ffa-9ae8-a280c0b8a86c None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:08.424299 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-d00e06d0-0938-4ffa-9ae8-a280c0b8a86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:08.453301 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:08.464447 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08633c7-4949-45bb-b490-ddf13f21d39d None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:08.507273 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44cd2c4-6229-4a23-8607-2ca04c0c0f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:08.508039 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44cd2c4-6229-4a23-8607-2ca04c0c0f9e None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:08.508421 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44cd2c4-6229-4a23-8607-2ca04c0c0f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:08.508717 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44cd2c4-6229-4a23-8607-2ca04c0c0f9e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:08.512790 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 88/165] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:15:08.525017 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44cd2c4-6229-4a23-8607-2ca04c0c0f9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:08.530577 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-d44cd2c4-6229-4a23-8607-2ca04c0c0f9e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:08.530577 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:08.530577 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:08.530577 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:08.530577 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:08.530577 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:08.530577 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:08.530577 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:08.530577 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:08.530577 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:08.530577 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:08.530577 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:08.530577 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:08.530577 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:08.530577 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:08.530577 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:08.531687 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:08.531687 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:08.531687 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:08.531687 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:08.531687 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:08.531687 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:08.531687 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:08.531687 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:08.531687 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:08.531687 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:08.531687 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:08.531687 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:08.531687 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:08.531687 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:08.531687 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:08.531687 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:08.532556 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:08.532556 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:08.532556 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:08.532556 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:08.532556 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:08.532556 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:08.532556 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:08.532556 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:08.532556 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:08.532556 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:08.532556 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:08.532556 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:08.534386 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 78/166] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:08.538786 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d59578-206b-49d6-9955-be0919ede35d None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:08.539247 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d59578-206b-49d6-9955-be0919ede35d None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:08.539667 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d59578-206b-49d6-9955-be0919ede35d None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:08.541724 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 89/167] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:08.542889 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44cd2c4-6229-4a23-8607-2ca04c0c0f9e None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:08.543646 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d59578-206b-49d6-9955-be0919ede35d None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:08.615773 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:08.653927 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44cd2c4-6229-4a23-8607-2ca04c0c0f9e None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:08.658791 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d59578-206b-49d6-9955-be0919ede35d None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:08.691688 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec9d72-9e8a-408a-a3b3-75ff7a0b774e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:08.692303 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec9d72-9e8a-408a-a3b3-75ff7a0b774e None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:08.692604 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec9d72-9e8a-408a-a3b3-75ff7a0b774e None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:08.692859 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec9d72-9e8a-408a-a3b3-75ff7a0b774e None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:08.693542 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dec9d72-9e8a-408a-a3b3-75ff7a0b774e None admin] RBAC: Adding query filter params (name=member) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:08.693828 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dec9d72-9e8a-408a-a3b3-75ff7a0b774e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:08.694539 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4cadc-7347-45ca-80c9-71145155a55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:08.695303 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dec9d72-9e8a-408a-a3b3-75ff7a0b774e None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:08.695642 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4cadc-7347-45ca-80c9-71145155a55e None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:08.696101 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4cadc-7347-45ca-80c9-71145155a55e None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:08.696764 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4cadc-7347-45ca-80c9-71145155a55e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:08.698065 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add4cadc-7347-45ca-80c9-71145155a55e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:08.699827 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add4cadc-7347-45ca-80c9-71145155a55e None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:08.701625 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 90/168] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:08.703004 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602b015-c52f-479d-99c5-5a3231845a09 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:08.703491 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602b015-c52f-479d-99c5-5a3231845a09 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:08.703996 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602b015-c52f-479d-99c5-5a3231845a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:08.708143 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 79/169] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 07:15:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:08.708948 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec9d72-9e8a-408a-a3b3-75ff7a0b774e None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:08.726767 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-a602b015-c52f-479d-99c5-5a3231845a09 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:08.728062 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-a602b015-c52f-479d-99c5-5a3231845a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:08.732670 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:08.797361 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 91/170] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:08.798035 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602b015-c52f-479d-99c5-5a3231845a09 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:08.820846 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec9d72-9e8a-408a-a3b3-75ff7a0b774e None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:08.855886 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6480db17-cb1f-4008-a03f-83383eb8138b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:08.856644 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480db17-cb1f-4008-a03f-83383eb8138b None admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:08.856924 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480db17-cb1f-4008-a03f-83383eb8138b None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:08.857176 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480db17-cb1f-4008-a03f-83383eb8138b None admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:08.857819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6480db17-cb1f-4008-a03f-83383eb8138b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:08.858647 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:08.859492 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6480db17-cb1f-4008-a03f-83383eb8138b None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:08.886707 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 80/171] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 07:15:08 2022] POST /identity/v3/users => generated 297 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:15:08.886707 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6480db17-cb1f-4008-a03f-83383eb8138b None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:08.915877 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602b015-c52f-479d-99c5-5a3231845a09 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:08.964969 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe9601-bedb-450d-8002-0a68bf0c7b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:08.966447 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe9601-bedb-450d-8002-0a68bf0c7b17 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:08.966890 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe9601-bedb-450d-8002-0a68bf0c7b17 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:08.967408 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe9601-bedb-450d-8002-0a68bf0c7b17 None admin] PATH_INFO: `/v3/users/802ecc46ca324e909fe9c176b3ffadfc` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:08.986013 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe9601-bedb-450d-8002-0a68bf0c7b17 None admin] RBAC: Authorizing `identity:get_user(user_id=802ecc46ca324e909fe9c176b3ffadfc)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:08.987025 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:08.988357 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe9601-bedb-450d-8002-0a68bf0c7b17 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:08.990595 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 92/172] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:08 2022] GET /identity/v3/users/802ecc46ca324e909fe9c176b3ffadfc => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:09.004705 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe9601-bedb-450d-8002-0a68bf0c7b17 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:09.016289 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6480db17-cb1f-4008-a03f-83383eb8138b None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:09.054385 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05322c63-d21d-425b-8093-68ce5ce5cada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:09.055129 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05322c63-d21d-425b-8093-68ce5ce5cada None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:09.055407 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05322c63-d21d-425b-8093-68ce5ce5cada None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:09.055799 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05322c63-d21d-425b-8093-68ce5ce5cada None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:09.070224 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05322c63-d21d-425b-8093-68ce5ce5cada None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:09.075902 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-05322c63-d21d-425b-8093-68ce5ce5cada None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:09.075902 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:09.075902 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:09.075902 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:09.075902 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:09.075902 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:09.075902 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:09.075902 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:09.075902 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:09.075902 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:09.075902 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:09.075902 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:09.075902 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:09.075902 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:09.075902 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:09.075902 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:09.076781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:09.076781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:09.076781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:09.076781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:09.076781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:09.076781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:09.076781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:09.076781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:09.076781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:09.076781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:09.076781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:09.076781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:09.076781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:09.076781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:09.076781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:09.076781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:09.077599 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:09.077599 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:09.077599 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:09.077599 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:09.077599 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:09.077599 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:09.077599 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:09.077599 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:09.077599 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:09.077599 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:09.077599 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:09.077599 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:09.077599 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 81/173] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 07:15:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:09.078249 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05322c63-d21d-425b-8093-68ce5ce5cada None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:09.142297 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe9601-bedb-450d-8002-0a68bf0c7b17 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:09.161117 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:09.184258 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f7707-6a72-4fd4-841a-5fbe7a17dc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:09.185165 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f7707-6a72-4fd4-841a-5fbe7a17dc53 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:09.185438 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f7707-6a72-4fd4-841a-5fbe7a17dc53 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:09.185708 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f7707-6a72-4fd4-841a-5fbe7a17dc53 None admin] PATH_INFO: `/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:09.189247 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7f7707-6a72-4fd4-841a-5fbe7a17dc53 None admin] RBAC: Authorizing `identity:get_project(project_id=ef9f41b2925a4825a22ae82a7bd2e955)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:09.192460 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7f7707-6a72-4fd4-841a-5fbe7a17dc53 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:09.193553 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 93/174] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 07:15:09 2022] GET /identity/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:09.194934 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f7707-6a72-4fd4-841a-5fbe7a17dc53 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:09.196726 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05322c63-d21d-425b-8093-68ce5ce5cada None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:09.240834 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58991d32-00a2-4836-88af-63b5e7d4239a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:09.242278 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58991d32-00a2-4836-88af-63b5e7d4239a None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:09.242560 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58991d32-00a2-4836-88af-63b5e7d4239a None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:09.242865 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58991d32-00a2-4836-88af-63b5e7d4239a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:09.245182 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:09.258223 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58991d32-00a2-4836-88af-63b5e7d4239a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:09.263325 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-58991d32-00a2-4836-88af-63b5e7d4239a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:15:09.263325 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:09.263325 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:09.263325 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:09.263325 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:09.263325 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:09.263325 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:09.263325 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:09.263325 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:09.263325 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:09.263325 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:09.263325 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:09.263325 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:09.263325 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:09.263325 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:09.263325 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:09.264263 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:09.264263 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:09.264263 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:09.264263 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:09.264263 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:09.264263 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:09.264263 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:09.264263 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:09.264263 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:09.264263 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:09.264263 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:09.264263 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:09.264263 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:09.264263 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:09.264263 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:09.264263 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:09.265034 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:09.265034 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:09.265034 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:09.265034 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:09.265034 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:09.265034 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:09.265034 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:09.265034 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:09.265034 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:09.265034 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:09.265034 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:15:09.265034 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:09.265034 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 82/175] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Nov 2 07:15:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:09.265681 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58991d32-00a2-4836-88af-63b5e7d4239a None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:09.319193 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f7707-6a72-4fd4-841a-5fbe7a17dc53 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:09.371134 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f3fb6-65c9-4d2d-8b2f-22938d4ea10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:09.378602 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f3fb6-65c9-4d2d-8b2f-22938d4ea10e None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:09.378602 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f3fb6-65c9-4d2d-8b2f-22938d4ea10e None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:09.378602 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f3fb6-65c9-4d2d-8b2f-22938d4ea10e None admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:09.378602 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f3fb6-65c9-4d2d-8b2f-22938d4ea10e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:09.378602 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f3fb6-65c9-4d2d-8b2f-22938d4ea10e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:09.378602 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f3fb6-65c9-4d2d-8b2f-22938d4ea10e None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:09.391054 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58991d32-00a2-4836-88af-63b5e7d4239a None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:09.395820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 94/176] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:09.396658 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f3fb6-65c9-4d2d-8b2f-22938d4ea10e None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:09.462575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db4f0fd-d0e5-4f1c-8338-4ced06755c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:09.463236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4f0fd-d0e5-4f1c-8338-4ced06755c24 None admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:09.464029 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4f0fd-d0e5-4f1c-8338-4ced06755c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:09.464029 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4f0fd-d0e5-4f1c-8338-4ced06755c24 None admin] PATH_INFO: `/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955/users/802ecc46ca324e909fe9c176b3ffadfc/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:09.469327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db4f0fd-d0e5-4f1c-8338-4ced06755c24 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef9f41b2925a4825a22ae82a7bd2e955, user_id=802ecc46ca324e909fe9c176b3ffadfc, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:09.470816 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db4f0fd-d0e5-4f1c-8338-4ced06755c24 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:09.488010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:09.529641 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 83/177] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 07:15:09 2022] PUT /identity/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955/users/802ecc46ca324e909fe9c176b3ffadfc/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:09.530943 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db4f0fd-d0e5-4f1c-8338-4ced06755c24 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:09.584776 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f3fb6-65c9-4d2d-8b2f-22938d4ea10e None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:09.667917 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:09.670289 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86978d6-d72e-4571-a6c1-7f3a9eb98405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:09.671758 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86978d6-d72e-4571-a6c1-7f3a9eb98405 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:09.675041 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86978d6-d72e-4571-a6c1-7f3a9eb98405 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:09.675041 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86978d6-d72e-4571-a6c1-7f3a9eb98405 None admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:09.675041 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86978d6-d72e-4571-a6c1-7f3a9eb98405 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:09.675041 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86978d6-d72e-4571-a6c1-7f3a9eb98405 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:09.679604 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86978d6-d72e-4571-a6c1-7f3a9eb98405 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:09.698864 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 95/178] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:09.709217 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86978d6-d72e-4571-a6c1-7f3a9eb98405 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:09.772705 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db4f0fd-d0e5-4f1c-8338-4ced06755c24 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:09.847568 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868747c-499a-4282-ae5c-e2750f84090d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:09.848567 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868747c-499a-4282-ae5c-e2750f84090d None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:09.848567 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868747c-499a-4282-ae5c-e2750f84090d None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:09.848788 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868747c-499a-4282-ae5c-e2750f84090d None admin] PATH_INFO: `/v3/users/280642cc66c84947b6bceab16f065990` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:09.850183 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7868747c-499a-4282-ae5c-e2750f84090d None admin] RBAC: Authorizing `identity:get_user(user_id=280642cc66c84947b6bceab16f065990)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:09.852354 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7868747c-499a-4282-ae5c-e2750f84090d None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:09.854338 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 84/179] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:09 2022] GET /identity/v3/users/280642cc66c84947b6bceab16f065990 => generated 283 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:09.874325 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868747c-499a-4282-ae5c-e2750f84090d None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:09.894746 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86978d6-d72e-4571-a6c1-7f3a9eb98405 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:09.908355 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:09.957797 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317c58a-8b01-4cea-acfd-cf8ba6102638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:09.958862 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317c58a-8b01-4cea-acfd-cf8ba6102638 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:09.959369 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317c58a-8b01-4cea-acfd-cf8ba6102638 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:09.959831 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317c58a-8b01-4cea-acfd-cf8ba6102638 None admin] PATH_INFO: `/v3/users/280642cc66c84947b6bceab16f065990` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:09.971876 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c317c58a-8b01-4cea-acfd-cf8ba6102638 None admin] RBAC: Authorizing `identity:get_user(user_id=280642cc66c84947b6bceab16f065990)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:09.974198 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c317c58a-8b01-4cea-acfd-cf8ba6102638 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:09.976133 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 96/180] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:09 2022] GET /identity/v3/users/280642cc66c84947b6bceab16f065990 => generated 283 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:09.987991 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317c58a-8b01-4cea-acfd-cf8ba6102638 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:10.174588 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317c58a-8b01-4cea-acfd-cf8ba6102638 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:10.184939 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868747c-499a-4282-ae5c-e2750f84090d None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:10.213128 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfdf321-b50e-46c9-b644-caa348787ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:10.213943 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfdf321-b50e-46c9-b644-caa348787ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:10.214221 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfdf321-b50e-46c9-b644-caa348787ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:10.214495 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfdf321-b50e-46c9-b644-caa348787ab8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:10.243029 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:10.268174 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:10.293678 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfdf321-b50e-46c9-b644-caa348787ab8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:10.295198 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfdf321-b50e-46c9-b644-caa348787ab8 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:10.303954 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28cd56d-fd26-491e-bd9b-2d2e15274776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:10.304591 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28cd56d-fd26-491e-bd9b-2d2e15274776 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:10.304869 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28cd56d-fd26-491e-bd9b-2d2e15274776 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:10.305136 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28cd56d-fd26-491e-bd9b-2d2e15274776 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:10.405565 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 97/181] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:15:10.411969 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfdf321-b50e-46c9-b644-caa348787ab8 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:10.492668 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28cd56d-fd26-491e-bd9b-2d2e15274776 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:10.494274 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28cd56d-fd26-491e-bd9b-2d2e15274776 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:10.522322 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:10.584874 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfdf321-b50e-46c9-b644-caa348787ab8 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:10.605699 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 85/182] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 732 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:15:10.611377 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28cd56d-fd26-491e-bd9b-2d2e15274776 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:10.636444 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:10.666836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97874c01-e517-493c-a97e-c741ea2fbc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:10.667486 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97874c01-e517-493c-a97e-c741ea2fbc6d None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:10.667764 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97874c01-e517-493c-a97e-c741ea2fbc6d None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:10.668044 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97874c01-e517-493c-a97e-c741ea2fbc6d None admin] PATH_INFO: `/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:10.669615 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97874c01-e517-493c-a97e-c741ea2fbc6d None admin] RBAC: Authorizing `identity:get_project(project_id=ef9f41b2925a4825a22ae82a7bd2e955)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:10.671539 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97874c01-e517-493c-a97e-c741ea2fbc6d None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:10.674219 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 98/183] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 07:15:10 2022] GET /identity/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955 => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:10.681560 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97874c01-e517-493c-a97e-c741ea2fbc6d None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:10.758635 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28cd56d-fd26-491e-bd9b-2d2e15274776 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:10.808856 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e425172a-b380-4976-8711-f471366ef334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:10.810030 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e425172a-b380-4976-8711-f471366ef334 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:10.810494 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e425172a-b380-4976-8711-f471366ef334 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:10.810992 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e425172a-b380-4976-8711-f471366ef334 None admin] PATH_INFO: `/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:10.812422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e425172a-b380-4976-8711-f471366ef334 None admin] RBAC: Authorizing `identity:get_project(project_id=ef9f41b2925a4825a22ae82a7bd2e955)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:10.814259 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e425172a-b380-4976-8711-f471366ef334 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:10.816166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 86/184] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 07:15:10 2022] GET /identity/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:10.827597 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e425172a-b380-4976-8711-f471366ef334 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:10.868105 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97874c01-e517-493c-a97e-c741ea2fbc6d None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:10.954114 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ae00b-88dc-44fc-bba1-f68545956e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:10.954756 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ae00b-88dc-44fc-bba1-f68545956e24 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:10.955044 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ae00b-88dc-44fc-bba1-f68545956e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:10.955316 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ae00b-88dc-44fc-bba1-f68545956e24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:10.956063 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8ae00b-88dc-44fc-bba1-f68545956e24 None admin] RBAC: Adding query filter params (role.id=39eab44f330440ebbefe39e538462165, scope.project.id=ef9f41b2925a4825a22ae82a7bd2e955, user.id=280642cc66c84947b6bceab16f065990) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:10.956349 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8ae00b-88dc-44fc-bba1-f68545956e24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:10.957806 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8ae00b-88dc-44fc-bba1-f68545956e24 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:10.963558 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:10.970648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 99/185] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 07:15:10 2022] GET /identity/v3/role_assignments?user.id=280642cc66c84947b6bceab16f065990&scope.project.id=ef9f41b2925a4825a22ae82a7bd2e955&role.id=39eab44f330440ebbefe39e538462165 => generated 608 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:10.976264 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e425172a-b380-4976-8711-f471366ef334 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:11.015412 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e6a915-1161-40df-9266-9d937e07ddea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:11.015946 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6a915-1161-40df-9266-9d937e07ddea None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:11.016220 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6a915-1161-40df-9266-9d937e07ddea None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:11.016489 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6a915-1161-40df-9266-9d937e07ddea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:11.017234 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e6a915-1161-40df-9266-9d937e07ddea None admin] RBAC: Adding query filter params (role.id=1e35bb725eb8475ea63e2d9538ab67d5, scope.project.id=ef9f41b2925a4825a22ae82a7bd2e955, user.id=280642cc66c84947b6bceab16f065990) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:11.017540 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e6a915-1161-40df-9266-9d937e07ddea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:11.019183 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e6a915-1161-40df-9266-9d937e07ddea None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:11.033140 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 87/186] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 07:15:10 2022] GET /identity/v3/role_assignments?user.id=280642cc66c84947b6bceab16f065990&scope.project.id=ef9f41b2925a4825a22ae82a7bd2e955&role.id=1e35bb725eb8475ea63e2d9538ab67d5 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:11.416613 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e107b-9729-4176-a41f-f804ab63ce74 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:11.416613 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e107b-9729-4176-a41f-f804ab63ce74 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:11.416613 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e107b-9729-4176-a41f-f804ab63ce74 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:11.417748 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 100/187] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:11.424034 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2cbe2-b828-484e-82bf-f5a6f8531c84 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:11.424993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2cbe2-b828-484e-82bf-f5a6f8531c84 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:11.425464 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2cbe2-b828-484e-82bf-f5a6f8531c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:11.444888 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-0dc2cbe2-b828-484e-82bf-f5a6f8531c84 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:11.446083 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-0dc2cbe2-b828-484e-82bf-f5a6f8531c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:11.515798 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 88/188] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:11.525384 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930402d9-51d4-49b3-875d-81c91e67e48f None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:11.525799 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930402d9-51d4-49b3-875d-81c91e67e48f None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:11.526066 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930402d9-51d4-49b3-875d-81c91e67e48f None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:11.527982 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 101/189] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:11.533202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafabdb6-40f4-4b22-9718-2411c06c6881 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:11.533447 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafabdb6-40f4-4b22-9718-2411c06c6881 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:11.533746 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafabdb6-40f4-4b22-9718-2411c06c6881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:11.552132 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-cafabdb6-40f4-4b22-9718-2411c06c6881 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:11.553306 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-cafabdb6-40f4-4b22-9718-2411c06c6881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:11.624597 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 89/190] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:11.783281 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930402d9-51d4-49b3-875d-81c91e67e48f None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:11.796710 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:11.886268 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafabdb6-40f4-4b22-9718-2411c06c6881 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:11.894137 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930402d9-51d4-49b3-875d-81c91e67e48f None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:11.903490 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:11.935063 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecdd60d-0df0-4fc9-91b8-6afb74192ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:11.935706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdd60d-0df0-4fc9-91b8-6afb74192ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:11.935984 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdd60d-0df0-4fc9-91b8-6afb74192ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:11.936255 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdd60d-0df0-4fc9-91b8-6afb74192ea1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:11.946494 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecdd60d-0df0-4fc9-91b8-6afb74192ea1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:11.951589 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-3ecdd60d-0df0-4fc9-91b8-6afb74192ea1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:15:11.951589 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:11.951589 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:11.951589 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:11.951589 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:11.951589 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:11.951589 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:11.951589 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:11.951589 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:11.951589 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:11.951589 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:11.951589 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:11.951589 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:11.951589 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:11.951589 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:11.951589 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:11.952626 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:11.952626 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:11.952626 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:11.952626 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:11.952626 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:11.952626 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:11.952626 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:11.952626 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:11.952626 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:11.952626 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:11.952626 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:11.952626 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:11.952626 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:11.952626 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:11.952626 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:11.952626 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:11.953523 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:11.953523 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:11.953523 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:11.953523 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:11.953523 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:11.953523 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:11.953523 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:11.953523 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:11.953523 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:11.953523 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:11.953523 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:15:11.953523 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:11.953523 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 102/191] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:11.958778 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecdd60d-0df0-4fc9-91b8-6afb74192ea1 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:12.001864 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafabdb6-40f4-4b22-9718-2411c06c6881 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:12.039037 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:12.041724 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac639cc4-c219-4324-bb2a-2e1b68b93b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:12.042396 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac639cc4-c219-4324-bb2a-2e1b68b93b60 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:12.042687 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac639cc4-c219-4324-bb2a-2e1b68b93b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:12.042958 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac639cc4-c219-4324-bb2a-2e1b68b93b60 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:12.044156 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac639cc4-c219-4324-bb2a-2e1b68b93b60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:12.045615 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac639cc4-c219-4324-bb2a-2e1b68b93b60 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:12.047433 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 90/192] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:12.052809 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac639cc4-c219-4324-bb2a-2e1b68b93b60 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:12.072649 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecdd60d-0df0-4fc9-91b8-6afb74192ea1 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:12.108022 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45802f-fd61-46e7-80fb-27ede14255a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:12.108703 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45802f-fd61-46e7-80fb-27ede14255a5 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:12.108892 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45802f-fd61-46e7-80fb-27ede14255a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:12.109141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45802f-fd61-46e7-80fb-27ede14255a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:12.109834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45802f-fd61-46e7-80fb-27ede14255a5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:12.110134 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45802f-fd61-46e7-80fb-27ede14255a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:12.111557 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45802f-fd61-46e7-80fb-27ede14255a5 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:12.124738 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 103/193] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 07:15:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:12.124738 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42a629-ace6-4e92-ba13-2a865c40c451 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:12.124738 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42a629-ace6-4e92-ba13-2a865c40c451 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:12.124738 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42a629-ace6-4e92-ba13-2a865c40c451 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:12.124738 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 104/194] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:12.127646 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42a629-ace6-4e92-ba13-2a865c40c451 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:12.142868 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:12.162957 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac639cc4-c219-4324-bb2a-2e1b68b93b60 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:12.196503 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e46d2c-4ab2-429d-9770-c2cd6f463bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:12.197107 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e46d2c-4ab2-429d-9770-c2cd6f463bb2 None admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:12.197521 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e46d2c-4ab2-429d-9770-c2cd6f463bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:12.197784 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e46d2c-4ab2-429d-9770-c2cd6f463bb2 None admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:12.198426 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e46d2c-4ab2-429d-9770-c2cd6f463bb2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:12.199905 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e46d2c-4ab2-429d-9770-c2cd6f463bb2 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:12.222357 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 91/195] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 07:15:12 2022] POST /identity/v3/users => generated 293 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:15:12.227585 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42a629-ace6-4e92-ba13-2a865c40c451 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:12.228398 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cffa93-e16d-4391-899c-e48adeb06c35 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:12.228744 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cffa93-e16d-4391-899c-e48adeb06c35 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:12.229018 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cffa93-e16d-4391-899c-e48adeb06c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:12.251778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-69cffa93-e16d-4391-899c-e48adeb06c35 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:12.253022 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-69cffa93-e16d-4391-899c-e48adeb06c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:12.262165 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:12.268154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510fb378-bcb3-4d0e-81f4-6717556b5d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:12.268839 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fb378-bcb3-4d0e-81f4-6717556b5d52 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:12.269113 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fb378-bcb3-4d0e-81f4-6717556b5d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:12.269382 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fb378-bcb3-4d0e-81f4-6717556b5d52 None admin] PATH_INFO: `/v3/users/75a7c06078ca41358be6a4ea64482ebf` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:12.287306 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510fb378-bcb3-4d0e-81f4-6717556b5d52 None admin] RBAC: Authorizing `identity:get_user(user_id=75a7c06078ca41358be6a4ea64482ebf)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:12.288947 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510fb378-bcb3-4d0e-81f4-6717556b5d52 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:12.291440 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 105/196] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:12 2022] GET /identity/v3/users/75a7c06078ca41358be6a4ea64482ebf => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:12.296639 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510fb378-bcb3-4d0e-81f4-6717556b5d52 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:12.341021 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 92/197] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:12.408279 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510fb378-bcb3-4d0e-81f4-6717556b5d52 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:12.442869 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e148b-762a-4311-8c00-271826545385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:12.443492 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e148b-762a-4311-8c00-271826545385 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:12.443773 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e148b-762a-4311-8c00-271826545385 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:12.444045 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e148b-762a-4311-8c00-271826545385 None admin] PATH_INFO: `/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:12.445171 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1e148b-762a-4311-8c00-271826545385 None admin] RBAC: Authorizing `identity:get_project(project_id=ef9f41b2925a4825a22ae82a7bd2e955)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:12.446625 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1e148b-762a-4311-8c00-271826545385 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:12.448604 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 106/198] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 07:15:12 2022] GET /identity/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:12.454528 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cffa93-e16d-4391-899c-e48adeb06c35 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:12.529508 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:12.565354 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cffa93-e16d-4391-899c-e48adeb06c35 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:12.605091 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922866cb-670a-4de0-90ee-2fe350cbce10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:12.605091 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922866cb-670a-4de0-90ee-2fe350cbce10 None admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:12.605091 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922866cb-670a-4de0-90ee-2fe350cbce10 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:12.605091 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922866cb-670a-4de0-90ee-2fe350cbce10 None admin] PATH_INFO: `/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955/users/75a7c06078ca41358be6a4ea64482ebf/roles/0ca4aff56e484fa99225a67bae03722f` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:12.606900 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922866cb-670a-4de0-90ee-2fe350cbce10 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef9f41b2925a4825a22ae82a7bd2e955, user_id=75a7c06078ca41358be6a4ea64482ebf, role_id=0ca4aff56e484fa99225a67bae03722f)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:12.608513 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922866cb-670a-4de0-90ee-2fe350cbce10 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:12.611489 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e148b-762a-4311-8c00-271826545385 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:12.629037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 93/199] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 07:15:12 2022] PUT /identity/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955/users/75a7c06078ca41358be6a4ea64482ebf/roles/0ca4aff56e484fa99225a67bae03722f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:12.688533 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:12.744167 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e148b-762a-4311-8c00-271826545385 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:12.784030 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78718497-1418-4f4a-9bb1-4c2608c4e81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:12.784694 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78718497-1418-4f4a-9bb1-4c2608c4e81e None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:12.784995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78718497-1418-4f4a-9bb1-4c2608c4e81e None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:12.785264 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78718497-1418-4f4a-9bb1-4c2608c4e81e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:12.797898 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78718497-1418-4f4a-9bb1-4c2608c4e81e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:12.803503 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-78718497-1418-4f4a-9bb1-4c2608c4e81e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:12.803503 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:12.803503 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:12.803503 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:12.803503 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:12.803503 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:12.803503 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:12.803503 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:12.803503 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:12.803503 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:12.803503 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:12.803503 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:12.803503 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:12.803503 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:12.803503 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:12.803503 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:12.804411 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:12.804411 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:12.804411 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:12.804411 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:12.804411 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:12.804411 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:12.804411 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:12.804411 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:12.804411 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:12.804411 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:12.804411 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:12.804411 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:12.804411 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:12.804411 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:12.804411 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:12.804411 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:12.805226 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:12.805226 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:12.805226 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:12.805226 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:12.805226 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:12.805226 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:12.805226 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:12.805226 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:12.805226 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:12.805226 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:12.805226 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:12.805226 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:12.805226 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 107/200] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:12.812832 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922866cb-670a-4de0-90ee-2fe350cbce10 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:12.918694 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922866cb-670a-4de0-90ee-2fe350cbce10 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:12.947106 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:12.956931 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d9a2a-171e-4d6d-95aa-bdb418a7faa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:12.957869 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d9a2a-171e-4d6d-95aa-bdb418a7faa8 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:12.958331 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d9a2a-171e-4d6d-95aa-bdb418a7faa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:12.959030 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d9a2a-171e-4d6d-95aa-bdb418a7faa8 None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:12.959899 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d9a2a-171e-4d6d-95aa-bdb418a7faa8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:12.960449 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d9a2a-171e-4d6d-95aa-bdb418a7faa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:12.962172 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d9a2a-171e-4d6d-95aa-bdb418a7faa8 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:12.971800 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 94/201] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 07:15:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:12.978320 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78718497-1418-4f4a-9bb1-4c2608c4e81e None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:13.081272 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78718497-1418-4f4a-9bb1-4c2608c4e81e None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:13.098811 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:13.116627 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a4dd0-b25a-42a9-8f7c-090ce1dd10b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:13.117280 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a4dd0-b25a-42a9-8f7c-090ce1dd10b3 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:13.117558 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a4dd0-b25a-42a9-8f7c-090ce1dd10b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:13.117830 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a4dd0-b25a-42a9-8f7c-090ce1dd10b3 None admin] PATH_INFO: `/v3/users/802ecc46ca324e909fe9c176b3ffadfc` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:13.119206 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a4dd0-b25a-42a9-8f7c-090ce1dd10b3 None admin] RBAC: Authorizing `identity:get_user(user_id=802ecc46ca324e909fe9c176b3ffadfc)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:13.120733 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a4dd0-b25a-42a9-8f7c-090ce1dd10b3 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:13.122527 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 108/202] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:12 2022] GET /identity/v3/users/802ecc46ca324e909fe9c176b3ffadfc => generated 297 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:13.128074 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d9a2a-171e-4d6d-95aa-bdb418a7faa8 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:13.236437 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d9a2a-171e-4d6d-95aa-bdb418a7faa8 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:13.270907 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b74786-cda4-49fd-b180-ab6ed0bbbeb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:13.271515 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b74786-cda4-49fd-b180-ab6ed0bbbeb9 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:13.271788 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b74786-cda4-49fd-b180-ab6ed0bbbeb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:13.272049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b74786-cda4-49fd-b180-ab6ed0bbbeb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:13.335050 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:13.344813 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b74786-cda4-49fd-b180-ab6ed0bbbeb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:13.346480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b74786-cda4-49fd-b180-ab6ed0bbbeb9 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:13.452389 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 95/203] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:15:13.458253 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a4dd0-b25a-42a9-8f7c-090ce1dd10b3 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:13.558580 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a4dd0-b25a-42a9-8f7c-090ce1dd10b3 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:13.602139 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d74e5-b5d7-4320-9f1a-d7f81f323620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:13.602827 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d74e5-b5d7-4320-9f1a-d7f81f323620 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:13.603132 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d74e5-b5d7-4320-9f1a-d7f81f323620 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:13.603419 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d74e5-b5d7-4320-9f1a-d7f81f323620 None admin] PATH_INFO: `/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:13.604565 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d74e5-b5d7-4320-9f1a-d7f81f323620 None admin] RBAC: Authorizing `identity:get_project(project_id=ef9f41b2925a4825a22ae82a7bd2e955)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:13.606044 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d74e5-b5d7-4320-9f1a-d7f81f323620 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:13.607995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 109/204] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 07:15:13 2022] GET /identity/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:13.613829 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b74786-cda4-49fd-b180-ab6ed0bbbeb9 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:13.711876 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:13.718047 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b74786-cda4-49fd-b180-ab6ed0bbbeb9 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:13.753660 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e42a63-aa29-475e-9027-0bea44bb145e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:13.754266 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e42a63-aa29-475e-9027-0bea44bb145e None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:13.754536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e42a63-aa29-475e-9027-0bea44bb145e None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:13.754826 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e42a63-aa29-475e-9027-0bea44bb145e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:13.755539 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e42a63-aa29-475e-9027-0bea44bb145e None admin] RBAC: Adding query filter params (role.id=15930dae1c0b464f98680edb03b7f02c, scope.project.id=ef9f41b2925a4825a22ae82a7bd2e955, user.id=802ecc46ca324e909fe9c176b3ffadfc) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:13.755836 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e42a63-aa29-475e-9027-0bea44bb145e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:13.757263 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e42a63-aa29-475e-9027-0bea44bb145e None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:13.764200 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 96/205] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 07:15:13 2022] GET /identity/v3/role_assignments?user.id=802ecc46ca324e909fe9c176b3ffadfc&scope.project.id=ef9f41b2925a4825a22ae82a7bd2e955&role.id=15930dae1c0b464f98680edb03b7f02c => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:14.610853 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf902967-270e-42f7-ad39-a70287f8400e None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:14.611321 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf902967-270e-42f7-ad39-a70287f8400e None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:14.611396 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf902967-270e-42f7-ad39-a70287f8400e None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:14.612707 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 110/206] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:14.618157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed43c54-7603-4b79-9927-6ef10173be8b None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:14.618575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed43c54-7603-4b79-9927-6ef10173be8b None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:14.618646 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed43c54-7603-4b79-9927-6ef10173be8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:14.637670 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-eed43c54-7603-4b79-9927-6ef10173be8b None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:14.638874 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-eed43c54-7603-4b79-9927-6ef10173be8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:14.678350 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef2ecc4-1b50-4777-8d16-1efa794aa832 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:14.678694 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef2ecc4-1b50-4777-8d16-1efa794aa832 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:14.678894 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef2ecc4-1b50-4777-8d16-1efa794aa832 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:14.680243 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 111/207] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:15:14.685373 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c50e22-626d-4344-ad35-f5ddf0d32ce4 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:14.685655 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c50e22-626d-4344-ad35-f5ddf0d32ce4 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:14.685948 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c50e22-626d-4344-ad35-f5ddf0d32ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:14.704024 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-42c50e22-626d-4344-ad35-f5ddf0d32ce4 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:14.705207 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-42c50e22-626d-4344-ad35-f5ddf0d32ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:14.708133 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 97/208] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:14.758552 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:14.780528 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 112/209] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:15:14.877056 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b8559-afb0-4a93-a28d-21704818e100 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:14.877234 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b8559-afb0-4a93-a28d-21704818e100 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:14.877490 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b8559-afb0-4a93-a28d-21704818e100 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:14.879032 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 98/210] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:14.884221 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7658ab6b-599f-4011-80a7-40633303769a None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:14.884508 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7658ab6b-599f-4011-80a7-40633303769a None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:14.884958 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7658ab6b-599f-4011-80a7-40633303769a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:14.904466 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-7658ab6b-599f-4011-80a7-40633303769a None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:14.905634 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-7658ab6b-599f-4011-80a7-40633303769a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:14.973354 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b8559-afb0-4a93-a28d-21704818e100 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:14.974174 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 113/211] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:15.051112 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7658ab6b-599f-4011-80a7-40633303769a None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:15.059545 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:15.080524 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b8559-afb0-4a93-a28d-21704818e100 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:15.116999 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a755a9-ac0e-40c9-ba71-f516cf347635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:15.117624 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a755a9-ac0e-40c9-ba71-f516cf347635 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:15.117898 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a755a9-ac0e-40c9-ba71-f516cf347635 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:15.118161 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a755a9-ac0e-40c9-ba71-f516cf347635 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:15.130258 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a755a9-ac0e-40c9-ba71-f516cf347635 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:15.135179 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-06a755a9-ac0e-40c9-ba71-f516cf347635 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:15.135179 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:15.135179 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:15.135179 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:15.135179 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:15.135179 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:15.135179 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:15.135179 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:15.135179 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:15.135179 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:15.135179 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:15.135179 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:15.135179 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:15.135179 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:15.135179 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:15.135179 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:15.136177 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:15.136177 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:15.136177 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:15.136177 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:15.136177 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:15.136177 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:15.136177 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:15.136177 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:15.136177 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:15.136177 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:15.136177 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:15.136177 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:15.136177 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:15.136177 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:15.136177 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:15.136177 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:15.136951 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:15.136951 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:15.136951 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:15.136951 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:15.136951 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:15.136951 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:15.136951 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:15.136951 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:15.136951 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:15.136951 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:15.136951 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:15.136951 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:15.136951 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 99/212] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:15.142057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a755a9-ac0e-40c9-ba71-f516cf347635 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:15.163011 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7658ab6b-599f-4011-80a7-40633303769a None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:15.177227 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:15.201803 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a14cbe-2adc-48f1-8f4d-02620f70f3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:15.202763 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a14cbe-2adc-48f1-8f4d-02620f70f3a5 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:15.203609 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a14cbe-2adc-48f1-8f4d-02620f70f3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:15.204075 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a14cbe-2adc-48f1-8f4d-02620f70f3a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:15.205747 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a14cbe-2adc-48f1-8f4d-02620f70f3a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:15.208021 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a14cbe-2adc-48f1-8f4d-02620f70f3a5 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:15.210839 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 114/213] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:15.216535 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a14cbe-2adc-48f1-8f4d-02620f70f3a5 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:15.252976 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a755a9-ac0e-40c9-ba71-f516cf347635 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:15.286064 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e637e1b-8443-4dc6-a6c1-7ffae966e16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:15.286902 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e637e1b-8443-4dc6-a6c1-7ffae966e16a None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:15.287181 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e637e1b-8443-4dc6-a6c1-7ffae966e16a None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:15.287537 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e637e1b-8443-4dc6-a6c1-7ffae966e16a None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:15.288212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e637e1b-8443-4dc6-a6c1-7ffae966e16a None admin] RBAC: Adding query filter params (name=member) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:15.288514 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e637e1b-8443-4dc6-a6c1-7ffae966e16a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:15.289902 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e637e1b-8443-4dc6-a6c1-7ffae966e16a None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:15.298509 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 100/214] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 07:15:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:15.299357 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e637e1b-8443-4dc6-a6c1-7ffae966e16a None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:15.308467 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:15.321004 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a14cbe-2adc-48f1-8f4d-02620f70f3a5 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:15.355089 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7022c715-0590-44f3-8bb1-d74509c6776b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:15.355687 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022c715-0590-44f3-8bb1-d74509c6776b None admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:15.355980 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022c715-0590-44f3-8bb1-d74509c6776b None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:15.356248 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022c715-0590-44f3-8bb1-d74509c6776b None admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:15.356897 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7022c715-0590-44f3-8bb1-d74509c6776b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:15.358347 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7022c715-0590-44f3-8bb1-d74509c6776b None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:15.379529 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 115/215] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 07:15:15 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:15:15.380467 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7022c715-0590-44f3-8bb1-d74509c6776b None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:15.398828 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e637e1b-8443-4dc6-a6c1-7ffae966e16a None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:15.437406 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:15.443911 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe1d147-7007-4835-bca8-64acb32a6b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:15.445074 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1d147-7007-4835-bca8-64acb32a6b05 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:15.445355 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1d147-7007-4835-bca8-64acb32a6b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:15.445622 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1d147-7007-4835-bca8-64acb32a6b05 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:15.457636 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe1d147-7007-4835-bca8-64acb32a6b05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:15.463255 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-ffe1d147-7007-4835-bca8-64acb32a6b05 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:15:15.463255 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:15.463255 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:15.463255 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:15.463255 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:15.463255 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:15.463255 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:15.463255 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:15.463255 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:15.463255 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:15.463255 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:15.463255 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:15.463255 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:15.463255 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:15.463255 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:15.463255 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:15.464212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:15.464212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:15.464212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:15.464212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:15.464212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:15.464212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:15.464212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:15.464212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:15.464212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:15.464212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:15.464212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:15.464212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:15.464212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:15.464212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:15.464212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:15.464212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:15.465000 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:15.465000 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:15.465000 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:15.465000 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:15.465000 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:15.465000 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:15.465000 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:15.465000 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:15.465000 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:15.465000 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:15.465000 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:15:15.465000 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:15.465000 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 101/216] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:15.470772 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe1d147-7007-4835-bca8-64acb32a6b05 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:15.512976 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7022c715-0590-44f3-8bb1-d74509c6776b None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:15.547554 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:15.551680 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0afa0dd-7069-46db-b494-8b077a17a3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:15.552349 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0afa0dd-7069-46db-b494-8b077a17a3e9 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:15.552621 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0afa0dd-7069-46db-b494-8b077a17a3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:15.552896 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0afa0dd-7069-46db-b494-8b077a17a3e9 None admin] PATH_INFO: `/v3/users/5ccec5a029414f8590cb1fb848d354a8` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:15.567724 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0afa0dd-7069-46db-b494-8b077a17a3e9 None admin] RBAC: Authorizing `identity:get_user(user_id=5ccec5a029414f8590cb1fb848d354a8)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:15.569278 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0afa0dd-7069-46db-b494-8b077a17a3e9 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:15.571060 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 116/217] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:15 2022] GET /identity/v3/users/5ccec5a029414f8590cb1fb848d354a8 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:15.577514 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0afa0dd-7069-46db-b494-8b077a17a3e9 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:15.593706 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe1d147-7007-4835-bca8-64acb32a6b05 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:15.637763 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155378b2-41ce-496e-ab2d-000eda15c050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:15.638359 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155378b2-41ce-496e-ab2d-000eda15c050 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:15.638359 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155378b2-41ce-496e-ab2d-000eda15c050 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:15.638509 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155378b2-41ce-496e-ab2d-000eda15c050 None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:15.639089 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155378b2-41ce-496e-ab2d-000eda15c050 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:15.639392 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155378b2-41ce-496e-ab2d-000eda15c050 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:15.640782 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155378b2-41ce-496e-ab2d-000eda15c050 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:15.650518 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 102/218] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 07:15:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:15.657244 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155378b2-41ce-496e-ab2d-000eda15c050 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:15.690476 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:15.698039 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0afa0dd-7069-46db-b494-8b077a17a3e9 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:15.737645 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13972e7-fbe4-494d-9193-42d2c200f099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:15.738259 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13972e7-fbe4-494d-9193-42d2c200f099 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:15.738531 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13972e7-fbe4-494d-9193-42d2c200f099 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:15.738833 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13972e7-fbe4-494d-9193-42d2c200f099 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:15.739559 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13972e7-fbe4-494d-9193-42d2c200f099 None admin] RBAC: Adding query filter params (role.id=15930dae1c0b464f98680edb03b7f02c, user.id=5ccec5a029414f8590cb1fb848d354a8, scope.system=all) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:15.739863 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13972e7-fbe4-494d-9193-42d2c200f099 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:15.741321 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13972e7-fbe4-494d-9193-42d2c200f099 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:15.748204 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 117/219] 10.222.0.30 () {56 vars in 1308 bytes} [Wed Nov 2 07:15:15 2022] GET /identity/v3/role_assignments?user.id=5ccec5a029414f8590cb1fb848d354a8&scope.system=all&role.id=15930dae1c0b464f98680edb03b7f02c => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:15.782995 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155378b2-41ce-496e-ab2d-000eda15c050 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:15.814028 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:15.824138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16099872-37db-4838-8774-4d6bbcbf4491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:15.824785 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16099872-37db-4838-8774-4d6bbcbf4491 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:15.825056 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16099872-37db-4838-8774-4d6bbcbf4491 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:15.825374 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16099872-37db-4838-8774-4d6bbcbf4491 None admin] PATH_INFO: `/v3/users/75a7c06078ca41358be6a4ea64482ebf` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:15.826606 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16099872-37db-4838-8774-4d6bbcbf4491 None admin] RBAC: Authorizing `identity:get_user(user_id=75a7c06078ca41358be6a4ea64482ebf)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:15.828097 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16099872-37db-4838-8774-4d6bbcbf4491 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:15.829902 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 103/220] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:15 2022] GET /identity/v3/users/75a7c06078ca41358be6a4ea64482ebf => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:15.835697 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13972e7-fbe4-494d-9193-42d2c200f099 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:15.957070 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13972e7-fbe4-494d-9193-42d2c200f099 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:15.993074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d088a-601f-4626-8a0a-35cf11ed5eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:15.993696 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d088a-601f-4626-8a0a-35cf11ed5eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:15.993971 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d088a-601f-4626-8a0a-35cf11ed5eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:15.994414 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d088a-601f-4626-8a0a-35cf11ed5eb1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:16.025960 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:16.069052 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d088a-601f-4626-8a0a-35cf11ed5eb1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:16.070534 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d088a-601f-4626-8a0a-35cf11ed5eb1 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:16.174966 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 118/221] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:15:16.182770 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16099872-37db-4838-8774-4d6bbcbf4491 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:16.285337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16099872-37db-4838-8774-4d6bbcbf4491 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:16.320754 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2412a2c1-cc49-4384-8219-3b7f45e1b6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:16.321394 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412a2c1-cc49-4384-8219-3b7f45e1b6ee None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:16.321668 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412a2c1-cc49-4384-8219-3b7f45e1b6ee None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:16.321940 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412a2c1-cc49-4384-8219-3b7f45e1b6ee None admin] PATH_INFO: `/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:16.323365 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2412a2c1-cc49-4384-8219-3b7f45e1b6ee None admin] RBAC: Authorizing `identity:get_project(project_id=ef9f41b2925a4825a22ae82a7bd2e955)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:16.324828 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2412a2c1-cc49-4384-8219-3b7f45e1b6ee None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:16.326493 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 104/222] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 07:15:16 2022] GET /identity/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:16.332503 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d088a-601f-4626-8a0a-35cf11ed5eb1 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:16.392855 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:16.435461 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d088a-601f-4626-8a0a-35cf11ed5eb1 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:16.475484 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd144a-39e5-47ed-80f1-eda93ea67ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:16.476457 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd144a-39e5-47ed-80f1-eda93ea67ef2 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:16.476925 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd144a-39e5-47ed-80f1-eda93ea67ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:16.477398 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd144a-39e5-47ed-80f1-eda93ea67ef2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:16.478334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bd144a-39e5-47ed-80f1-eda93ea67ef2 None admin] RBAC: Adding query filter params (role.id=0ca4aff56e484fa99225a67bae03722f, scope.project.id=ef9f41b2925a4825a22ae82a7bd2e955, user.id=75a7c06078ca41358be6a4ea64482ebf) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:16.479023 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bd144a-39e5-47ed-80f1-eda93ea67ef2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:16.480675 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bd144a-39e5-47ed-80f1-eda93ea67ef2 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:16.487164 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 119/223] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 07:15:16 2022] GET /identity/v3/role_assignments?user.id=75a7c06078ca41358be6a4ea64482ebf&scope.project.id=ef9f41b2925a4825a22ae82a7bd2e955&role.id=0ca4aff56e484fa99225a67bae03722f => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:17.698116 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e2585-cc0e-4d8a-be9c-545694137133 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:17.698912 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e2585-cc0e-4d8a-be9c-545694137133 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:17.698912 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e2585-cc0e-4d8a-be9c-545694137133 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:17.700278 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 105/224] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:17.706214 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be88b7-8788-4297-bdf1-0eb0659507df None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:17.706816 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be88b7-8788-4297-bdf1-0eb0659507df None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:17.706816 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be88b7-8788-4297-bdf1-0eb0659507df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:17.725813 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-e7be88b7-8788-4297-bdf1-0eb0659507df None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:17.727041 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-e7be88b7-8788-4297-bdf1-0eb0659507df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:17.800612 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93650028-658b-4635-b107-5a714c4d9b52 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:17.800968 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93650028-658b-4635-b107-5a714c4d9b52 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:17.801165 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93650028-658b-4635-b107-5a714c4d9b52 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:17.801367 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 120/225] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:17.802511 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 106/226] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:17.808012 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409038a-cb78-4388-afb2-f010acab2608 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:17.808495 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409038a-cb78-4388-afb2-f010acab2608 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:17.808776 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409038a-cb78-4388-afb2-f010acab2608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:17.828278 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-e409038a-cb78-4388-afb2-f010acab2608 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:17.829486 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-e409038a-cb78-4388-afb2-f010acab2608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:17.885944 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:17.905816 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 121/227] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:17.980927 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88463d3a-8397-4d0a-b092-e0b0516fddeb None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:17.981265 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88463d3a-8397-4d0a-b092-e0b0516fddeb None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:17.981466 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88463d3a-8397-4d0a-b092-e0b0516fddeb None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:17.983031 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 107/228] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:17.988188 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c1a94-e4ab-4755-bbe7-f0630f873088 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:17.988489 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c1a94-e4ab-4755-bbe7-f0630f873088 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:17.988742 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c1a94-e4ab-4755-bbe7-f0630f873088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:18.007488 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-7e8c1a94-e4ab-4755-bbe7-f0630f873088 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:18.008671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-7e8c1a94-e4ab-4755-bbe7-f0630f873088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:18.068222 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88463d3a-8397-4d0a-b092-e0b0516fddeb None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:18.079770 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 122/229] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:18.133798 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:18.171372 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c1a94-e4ab-4755-bbe7-f0630f873088 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:18.177876 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88463d3a-8397-4d0a-b092-e0b0516fddeb None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:18.215269 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa0aa0-020d-4ed2-977c-7937ead16b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:18.215925 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa0aa0-020d-4ed2-977c-7937ead16b0b None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:18.216229 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa0aa0-020d-4ed2-977c-7937ead16b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:18.216526 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa0aa0-020d-4ed2-977c-7937ead16b0b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:18.229013 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fa0aa0-020d-4ed2-977c-7937ead16b0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:18.234262 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-52fa0aa0-020d-4ed2-977c-7937ead16b0b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:15:18.234262 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:18.234262 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:18.234262 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:18.234262 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:18.234262 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:18.234262 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:18.234262 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:18.234262 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:18.234262 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:18.234262 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:18.234262 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:18.234262 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:18.234262 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:18.234262 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:18.234262 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:18.235400 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:18.235400 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:18.235400 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:18.235400 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:18.235400 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:18.235400 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:18.235400 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:18.235400 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:18.235400 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:18.235400 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:18.235400 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:18.235400 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:18.235400 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:18.235400 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:18.235400 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:18.235400 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:18.236385 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:18.236385 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:18.236385 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:18.236385 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:18.236385 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:18.236385 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:18.236385 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:18.236385 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:18.236385 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:18.236385 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:18.236385 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:15:18.236385 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:18.236385 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 108/230] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:18.243222 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa0aa0-020d-4ed2-977c-7937ead16b0b None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:18.283807 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c1a94-e4ab-4755-bbe7-f0630f873088 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:18.299267 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:18.320221 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff447a6-6fc7-41ec-b5c4-bc0fdcc0358a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:18.320852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff447a6-6fc7-41ec-b5c4-bc0fdcc0358a None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:18.321126 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff447a6-6fc7-41ec-b5c4-bc0fdcc0358a None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:18.321393 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff447a6-6fc7-41ec-b5c4-bc0fdcc0358a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:18.322503 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff447a6-6fc7-41ec-b5c4-bc0fdcc0358a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:18.323989 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff447a6-6fc7-41ec-b5c4-bc0fdcc0358a None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:18.325555 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 123/231] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:18.331101 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff447a6-6fc7-41ec-b5c4-bc0fdcc0358a None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:18.349856 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa0aa0-020d-4ed2-977c-7937ead16b0b None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:18.372640 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:18.383291 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d69b95-baea-4750-b9a6-ba549cadd72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:18.383892 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d69b95-baea-4750-b9a6-ba549cadd72f None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:18.384218 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d69b95-baea-4750-b9a6-ba549cadd72f None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:18.384486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d69b95-baea-4750-b9a6-ba549cadd72f None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:18.385288 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d69b95-baea-4750-b9a6-ba549cadd72f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:18.385663 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d69b95-baea-4750-b9a6-ba549cadd72f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:18.387085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d69b95-baea-4750-b9a6-ba549cadd72f None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:18.395086 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 109/232] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 07:15:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:18.397317 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d69b95-baea-4750-b9a6-ba549cadd72f None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:18.430036 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff447a6-6fc7-41ec-b5c4-bc0fdcc0358a None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:18.463717 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da579354-a8ea-4ecd-8a83-f4946418ad06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:18.464335 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da579354-a8ea-4ecd-8a83-f4946418ad06 None admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:18.464613 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da579354-a8ea-4ecd-8a83-f4946418ad06 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:18.464884 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da579354-a8ea-4ecd-8a83-f4946418ad06 None admin] PATH_INFO: `/v3/groups` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:18.465512 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da579354-a8ea-4ecd-8a83-f4946418ad06 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:18.466997 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da579354-a8ea-4ecd-8a83-f4946418ad06 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:18.483780 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 124/233] 10.222.0.30 () {60 vars in 1134 bytes} [Wed Nov 2 07:15:18 2022] POST /identity/v3/groups => generated 229 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:15:18.490972 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da579354-a8ea-4ecd-8a83-f4946418ad06 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:18.498674 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d69b95-baea-4750-b9a6-ba549cadd72f None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:18.536781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7215f-7eb5-4921-8460-43832138f630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:18.537432 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7215f-7eb5-4921-8460-43832138f630 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:18.537708 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7215f-7eb5-4921-8460-43832138f630 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:18.537976 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7215f-7eb5-4921-8460-43832138f630 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:18.541528 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:18.550863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a7215f-7eb5-4921-8460-43832138f630 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:18.556488 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-f7a7215f-7eb5-4921-8460-43832138f630 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:18.556488 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:18.556488 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:18.556488 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:18.556488 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:18.556488 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:18.556488 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:18.556488 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:18.556488 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:18.556488 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:18.556488 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:18.556488 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:18.556488 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:18.556488 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:18.556488 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:18.556488 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:18.557486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:18.557486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:18.557486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:18.557486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:18.557486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:18.557486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:18.557486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:18.557486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:18.557486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:18.557486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:18.557486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:18.557486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:18.557486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:18.557486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:18.557486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:18.557486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:18.558270 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:18.558270 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:18.558270 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:18.558270 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:18.558270 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:18.558270 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:18.558270 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:18.558270 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:18.558270 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:18.558270 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:18.558270 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:18.558270 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:18.558270 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 110/234] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:18.564028 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7215f-7eb5-4921-8460-43832138f630 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:18.621090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:18.623376 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da579354-a8ea-4ecd-8a83-f4946418ad06 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:18.664083 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef85c45-80cb-4037-ae24-6da022507000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:18.664734 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef85c45-80cb-4037-ae24-6da022507000 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:18.665220 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef85c45-80cb-4037-ae24-6da022507000 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:18.665338 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef85c45-80cb-4037-ae24-6da022507000 None admin] PATH_INFO: `/v3/users/41a6cc02725648459037ee20691db41c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:18.680162 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef85c45-80cb-4037-ae24-6da022507000 None admin] RBAC: Authorizing `identity:get_user(user_id=41a6cc02725648459037ee20691db41c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:18.681605 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef85c45-80cb-4037-ae24-6da022507000 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:18.683887 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 125/235] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:18 2022] GET /identity/v3/users/41a6cc02725648459037ee20691db41c => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:18.687318 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7215f-7eb5-4921-8460-43832138f630 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:18.689625 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef85c45-80cb-4037-ae24-6da022507000 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:18.722489 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a3cfa-e6da-4ba2-bf15-00c831bc71b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:18.723273 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a3cfa-e6da-4ba2-bf15-00c831bc71b7 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:18.723701 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a3cfa-e6da-4ba2-bf15-00c831bc71b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:18.723967 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a3cfa-e6da-4ba2-bf15-00c831bc71b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:18.724630 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a3cfa-e6da-4ba2-bf15-00c831bc71b7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:18.724931 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a3cfa-e6da-4ba2-bf15-00c831bc71b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:18.726322 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a3cfa-e6da-4ba2-bf15-00c831bc71b7 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:18.735315 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 111/236] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 07:15:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:18.740633 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a3cfa-e6da-4ba2-bf15-00c831bc71b7 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:18.793065 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:18.796771 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef85c45-80cb-4037-ae24-6da022507000 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:18.839001 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c0d6e-b4d0-41a8-b187-5605214a1b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:18.839871 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c0d6e-b4d0-41a8-b187-5605214a1b46 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:18.839871 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c0d6e-b4d0-41a8-b187-5605214a1b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:18.840167 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c0d6e-b4d0-41a8-b187-5605214a1b46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:18.840888 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c0d6e-b4d0-41a8-b187-5605214a1b46 None admin] RBAC: Adding query filter params (role.id=0ca4aff56e484fa99225a67bae03722f, user.id=41a6cc02725648459037ee20691db41c, scope.system=all) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:18.841198 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c0d6e-b4d0-41a8-b187-5605214a1b46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:18.842651 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c0d6e-b4d0-41a8-b187-5605214a1b46 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:18.846560 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a3cfa-e6da-4ba2-bf15-00c831bc71b7 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:18.847905 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 126/237] 10.222.0.30 () {56 vars in 1308 bytes} [Wed Nov 2 07:15:18 2022] GET /identity/v3/role_assignments?user.id=41a6cc02725648459037ee20691db41c&scope.system=all&role.id=0ca4aff56e484fa99225a67bae03722f => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:18.857214 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:18.887886 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682bd631-eda4-4805-b295-bb3257e1c498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:18.888547 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682bd631-eda4-4805-b295-bb3257e1c498 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:18.888820 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682bd631-eda4-4805-b295-bb3257e1c498 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:18.889085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682bd631-eda4-4805-b295-bb3257e1c498 None admin] PATH_INFO: `/v3/users/5ccec5a029414f8590cb1fb848d354a8` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:18.890287 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682bd631-eda4-4805-b295-bb3257e1c498 None admin] RBAC: Authorizing `identity:get_user(user_id=5ccec5a029414f8590cb1fb848d354a8)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:18.891816 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682bd631-eda4-4805-b295-bb3257e1c498 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:18.893722 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 112/238] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:18 2022] GET /identity/v3/users/5ccec5a029414f8590cb1fb848d354a8 => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:18.899494 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c0d6e-b4d0-41a8-b187-5605214a1b46 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:19.012349 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c0d6e-b4d0-41a8-b187-5605214a1b46 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:19.050212 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d1127-2f0b-4843-85c5-bcc4eaaf6b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:19.050925 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d1127-2f0b-4843-85c5-bcc4eaaf6b08 None admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:19.051204 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d1127-2f0b-4843-85c5-bcc4eaaf6b08 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:19.051476 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d1127-2f0b-4843-85c5-bcc4eaaf6b08 None admin] PATH_INFO: `/v3/system/users/5ccec5a029414f8590cb1fb848d354a8/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:19.053007 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d1127-2f0b-4843-85c5-bcc4eaaf6b08 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5ccec5a029414f8590cb1fb848d354a8, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:19.054417 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d1127-2f0b-4843-85c5-bcc4eaaf6b08 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:19.064952 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 127/239] 10.222.0.30 () {58 vars in 1334 bytes} [Wed Nov 2 07:15:18 2022] PUT /identity/v3/system/users/5ccec5a029414f8590cb1fb848d354a8/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:20.653019 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c82f8a6-af23-407a-97d6-860476a9fecb None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:20.653908 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c82f8a6-af23-407a-97d6-860476a9fecb None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:20.653908 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c82f8a6-af23-407a-97d6-860476a9fecb None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:20.655318 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 113/240] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:20.660865 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b31bd-5207-493a-9bfe-de538db780cf None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:20.661308 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b31bd-5207-493a-9bfe-de538db780cf None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:20.661379 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b31bd-5207-493a-9bfe-de538db780cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:20.680411 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-2b9b31bd-5207-493a-9bfe-de538db780cf None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:20.681580 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-2b9b31bd-5207-493a-9bfe-de538db780cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:20.705364 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:20.753549 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 128/241] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:15:20.984918 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9d235-53b8-4dd4-b2bc-14e0ac576e45 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:20.985114 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9d235-53b8-4dd4-b2bc-14e0ac576e45 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:20.985397 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9d235-53b8-4dd4-b2bc-14e0ac576e45 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:20.986891 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 114/242] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:20.992032 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6243b-7dcd-458d-9eed-2c6e272f33cb None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:20.992315 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6243b-7dcd-458d-9eed-2c6e272f33cb None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:20.992584 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6243b-7dcd-458d-9eed-2c6e272f33cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:21.011834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-eef6243b-7dcd-458d-9eed-2c6e272f33cb None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:21.013029 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-eef6243b-7dcd-458d-9eed-2c6e272f33cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:21.017863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9d235-53b8-4dd4-b2bc-14e0ac576e45 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:21.090430 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 129/243] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:21.128953 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9d235-53b8-4dd4-b2bc-14e0ac576e45 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:21.163510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a071496-36b0-4b9e-a706-0e7ae8b498ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:21.164141 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a071496-36b0-4b9e-a706-0e7ae8b498ab None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:21.164414 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a071496-36b0-4b9e-a706-0e7ae8b498ab None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:21.164676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a071496-36b0-4b9e-a706-0e7ae8b498ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:21.165777 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a071496-36b0-4b9e-a706-0e7ae8b498ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:21.167300 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a071496-36b0-4b9e-a706-0e7ae8b498ab None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:21.169103 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 115/244] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:21.174504 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6243b-7dcd-458d-9eed-2c6e272f33cb None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:21.198296 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2af04-64de-4525-b0fc-ef6b3fd5d6f4 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:21.198635 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2af04-64de-4525-b0fc-ef6b3fd5d6f4 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:21.198859 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2af04-64de-4525-b0fc-ef6b3fd5d6f4 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:21.200198 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 116/245] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:15:21.205052 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91866734-3067-4db0-b8e6-c80ebe9349dc None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:21.205328 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91866734-3067-4db0-b8e6-c80ebe9349dc None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:21.205598 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91866734-3067-4db0-b8e6-c80ebe9349dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:21.224350 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-91866734-3067-4db0-b8e6-c80ebe9349dc None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:21.225814 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-91866734-3067-4db0-b8e6-c80ebe9349dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:21.267677 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:21.275633 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:21.281697 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6243b-7dcd-458d-9eed-2c6e272f33cb None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:21.306260 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 117/246] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:15:21.315623 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d3e76-3bb4-415d-8f54-7986a3a644b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:21.316286 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d3e76-3bb4-415d-8f54-7986a3a644b2 None admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:21.316557 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d3e76-3bb4-415d-8f54-7986a3a644b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:21.316818 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d3e76-3bb4-415d-8f54-7986a3a644b2 None admin] PATH_INFO: `/v3/groups` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:21.317490 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d3e76-3bb4-415d-8f54-7986a3a644b2 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:21.319009 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d3e76-3bb4-415d-8f54-7986a3a644b2 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:21.342683 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 130/247] 10.222.0.30 () {60 vars in 1134 bytes} [Wed Nov 2 07:15:21 2022] POST /identity/v3/groups => generated 226 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:15:21.357034 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91866734-3067-4db0-b8e6-c80ebe9349dc None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:21.471543 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91866734-3067-4db0-b8e6-c80ebe9349dc None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:21.509025 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad12ca8a-5b9a-4529-8ec0-639357a1c7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:21.509667 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12ca8a-5b9a-4529-8ec0-639357a1c7f3 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:21.509943 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12ca8a-5b9a-4529-8ec0-639357a1c7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:21.510208 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12ca8a-5b9a-4529-8ec0-639357a1c7f3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:21.521375 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad12ca8a-5b9a-4529-8ec0-639357a1c7f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:21.526395 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-ad12ca8a-5b9a-4529-8ec0-639357a1c7f3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:15:21.526395 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:21.526395 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:21.526395 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:21.526395 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:21.526395 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:21.526395 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:21.526395 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:21.526395 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:21.526395 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:21.526395 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:21.526395 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:21.526395 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:21.526395 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:21.526395 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:21.526395 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:21.528057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:21.528057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:21.528057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:21.528057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:21.528057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:21.528057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:21.528057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:21.528057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:21.528057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:21.528057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:21.528057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:21.528057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:21.528057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:21.528057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:21.528057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:21.528057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:21.529411 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:21.529411 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:21.529411 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:21.529411 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:21.529411 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:21.529411 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:21.529411 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:21.529411 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:21.529411 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:21.529411 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:21.529411 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:15:21.529411 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:21.529411 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 118/248] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:21.533782 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d3e76-3bb4-415d-8f54-7986a3a644b2 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:21.575030 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad12ca8a-5b9a-4529-8ec0-639357a1c7f3 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:21.621340 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:21.649875 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d3e76-3bb4-415d-8f54-7986a3a644b2 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:21.693056 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f518b4-296b-4a0f-a52f-95f419b0c816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:21.693674 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f518b4-296b-4a0f-a52f-95f419b0c816 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:21.693933 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f518b4-296b-4a0f-a52f-95f419b0c816 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:21.694202 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f518b4-296b-4a0f-a52f-95f419b0c816 None admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:21.694897 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f518b4-296b-4a0f-a52f-95f419b0c816 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:21.695194 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f518b4-296b-4a0f-a52f-95f419b0c816 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:21.696584 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f518b4-296b-4a0f-a52f-95f419b0c816 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:21.700559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad12ca8a-5b9a-4529-8ec0-639357a1c7f3 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:21.709182 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 131/249] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 07:15:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:21.715213 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f518b4-296b-4a0f-a52f-95f419b0c816 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:21.734688 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:21.745919 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d650d1-777b-4906-bedd-9e6c9675ddb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:21.746592 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d650d1-777b-4906-bedd-9e6c9675ddb4 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:21.746973 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d650d1-777b-4906-bedd-9e6c9675ddb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:21.750163 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d650d1-777b-4906-bedd-9e6c9675ddb4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:21.765224 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d650d1-777b-4906-bedd-9e6c9675ddb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:21.771529 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-02d650d1-777b-4906-bedd-9e6c9675ddb4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:21.771529 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:21.771529 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:21.771529 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:21.771529 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:21.771529 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:21.771529 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:21.771529 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:21.771529 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:21.771529 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:21.771529 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:21.771529 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:21.771529 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:21.771529 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:21.771529 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:21.771529 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:21.772381 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:21.772381 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:21.772381 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:21.772381 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:21.772381 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:21.772381 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:21.772381 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:21.772381 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:21.772381 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:21.772381 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:21.772381 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:21.772381 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:21.772381 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:21.772381 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:21.772381 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:21.772381 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:21.773176 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:21.773176 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:21.773176 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:21.773176 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:21.773176 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:21.773176 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:21.773176 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:21.773176 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:21.773176 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:21.773176 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:21.773176 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:21.773176 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:21.774273 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 119/250] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:21.780240 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d650d1-777b-4906-bedd-9e6c9675ddb4 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:21.849816 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f518b4-296b-4a0f-a52f-95f419b0c816 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:21.888004 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10157a89-c304-4056-be28-573ee35677c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:21.888691 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10157a89-c304-4056-be28-573ee35677c4 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:21.889097 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10157a89-c304-4056-be28-573ee35677c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:21.889511 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10157a89-c304-4056-be28-573ee35677c4 None admin] PATH_INFO: `/v3/users/41a6cc02725648459037ee20691db41c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:21.891435 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10157a89-c304-4056-be28-573ee35677c4 None admin] RBAC: Authorizing `identity:get_user(user_id=41a6cc02725648459037ee20691db41c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:21.893210 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10157a89-c304-4056-be28-573ee35677c4 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:21.895149 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 132/251] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:21 2022] GET /identity/v3/users/41a6cc02725648459037ee20691db41c => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:21.902021 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10157a89-c304-4056-be28-573ee35677c4 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:21.905827 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:21.914524 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d650d1-777b-4906-bedd-9e6c9675ddb4 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:21.964111 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0d84d-a81c-4cc8-9952-d465ca393ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:21.964737 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0d84d-a81c-4cc8-9952-d465ca393ce0 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:21.965014 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0d84d-a81c-4cc8-9952-d465ca393ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:21.965283 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0d84d-a81c-4cc8-9952-d465ca393ce0 None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:21.966022 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf0d84d-a81c-4cc8-9952-d465ca393ce0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:21.966247 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf0d84d-a81c-4cc8-9952-d465ca393ce0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:21.969451 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf0d84d-a81c-4cc8-9952-d465ca393ce0 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:21.980087 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 120/252] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 07:15:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:21.985941 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0d84d-a81c-4cc8-9952-d465ca393ce0 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:22.012821 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:22.048403 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10157a89-c304-4056-be28-573ee35677c4 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:22.100577 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad626e4c-10ba-42dc-adf2-e51ff236d53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:22.102192 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad626e4c-10ba-42dc-adf2-e51ff236d53a None admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:22.102956 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad626e4c-10ba-42dc-adf2-e51ff236d53a None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:22.103409 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad626e4c-10ba-42dc-adf2-e51ff236d53a None admin] PATH_INFO: `/v3/system/users/41a6cc02725648459037ee20691db41c/roles/0ca4aff56e484fa99225a67bae03722f` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:22.104712 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0d84d-a81c-4cc8-9952-d465ca393ce0 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:22.105353 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad626e4c-10ba-42dc-adf2-e51ff236d53a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=41a6cc02725648459037ee20691db41c, role_id=0ca4aff56e484fa99225a67bae03722f)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:22.107261 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad626e4c-10ba-42dc-adf2-e51ff236d53a None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:22.126955 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 133/253] 10.222.0.30 () {58 vars in 1334 bytes} [Wed Nov 2 07:15:21 2022] PUT /identity/v3/system/users/41a6cc02725648459037ee20691db41c/roles/0ca4aff56e484fa99225a67bae03722f => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:22.151258 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd7b42-6192-4788-a3ef-2589059cb33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:22.151989 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd7b42-6192-4788-a3ef-2589059cb33a None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:22.152414 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd7b42-6192-4788-a3ef-2589059cb33a None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:22.152686 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd7b42-6192-4788-a3ef-2589059cb33a None admin] PATH_INFO: `/v3/users/5ccec5a029414f8590cb1fb848d354a8` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:22.154265 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dd7b42-6192-4788-a3ef-2589059cb33a None admin] RBAC: Authorizing `identity:get_user(user_id=5ccec5a029414f8590cb1fb848d354a8)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:22.155815 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dd7b42-6192-4788-a3ef-2589059cb33a None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:22.157755 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 121/254] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:21 2022] GET /identity/v3/users/5ccec5a029414f8590cb1fb848d354a8 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:22.164050 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad626e4c-10ba-42dc-adf2-e51ff236d53a None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:22.282790 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad626e4c-10ba-42dc-adf2-e51ff236d53a None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:22.329455 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8291c8f-2c5b-44ea-81a5-be773807395f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:22.329993 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8291c8f-2c5b-44ea-81a5-be773807395f None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:22.330311 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8291c8f-2c5b-44ea-81a5-be773807395f None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:22.330581 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8291c8f-2c5b-44ea-81a5-be773807395f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:22.331371 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8291c8f-2c5b-44ea-81a5-be773807395f None admin] RBAC: Adding query filter params (role.id=15930dae1c0b464f98680edb03b7f02c, user.id=5ccec5a029414f8590cb1fb848d354a8, scope.system=all) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:22.331672 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8291c8f-2c5b-44ea-81a5-be773807395f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:22.333196 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8291c8f-2c5b-44ea-81a5-be773807395f None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:22.348424 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:22.353244 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 134/255] 10.222.0.30 () {56 vars in 1308 bytes} [Wed Nov 2 07:15:22 2022] GET /identity/v3/role_assignments?user.id=5ccec5a029414f8590cb1fb848d354a8&scope.system=all&role.id=15930dae1c0b464f98680edb03b7f02c => generated 510 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:23.955972 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f161f-2480-42e0-99b5-c17a978e06aa None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:23.956561 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f161f-2480-42e0-99b5-c17a978e06aa None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:23.956561 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f161f-2480-42e0-99b5-c17a978e06aa None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:23.958000 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 122/256] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:23.967637 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3b172-993b-4b66-8b02-c414d6a84b4e None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:23.968111 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3b172-993b-4b66-8b02-c414d6a84b4e None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:23.968184 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3b172-993b-4b66-8b02-c414d6a84b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:23.999719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-a2b3b172-993b-4b66-8b02-c414d6a84b4e None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:24.000923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-a2b3b172-993b-4b66-8b02-c414d6a84b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:24.077837 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 135/257] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Nov 02 07:15:24.396577 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08d367-b2d7-4ed9-b3b0-52a1299dee64 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:24.396958 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08d367-b2d7-4ed9-b3b0-52a1299dee64 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:24.397149 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08d367-b2d7-4ed9-b3b0-52a1299dee64 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:24.398716 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 123/258] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:24.413553 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323bb27d-1468-43e7-9b96-606370202964 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:24.413834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323bb27d-1468-43e7-9b96-606370202964 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:24.414092 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323bb27d-1468-43e7-9b96-606370202964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:24.434854 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08d367-b2d7-4ed9-b3b0-52a1299dee64 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:24.439716 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-323bb27d-1468-43e7-9b96-606370202964 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:24.440906 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-323bb27d-1468-43e7-9b96-606370202964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:24.448423 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08d367-b2d7-4ed9-b3b0-52a1299dee64 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:24.451099 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 136/259] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:24 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Nov 02 07:15:24.452820 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2802ef-9b45-4b55-8ea1-3ad931ea3ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:24.453462 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2802ef-9b45-4b55-8ea1-3ad931ea3ec7 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:24.453739 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2802ef-9b45-4b55-8ea1-3ad931ea3ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:24.454014 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2802ef-9b45-4b55-8ea1-3ad931ea3ec7 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:24.467067 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2802ef-9b45-4b55-8ea1-3ad931ea3ec7 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:24.471863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-fd2802ef-9b45-4b55-8ea1-3ad931ea3ec7 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:15:24.471863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:24.471863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:24.471863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:24.471863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:24.471863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:24.471863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:24.471863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:24.471863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:24.471863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:24.471863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:24.471863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:24.471863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:24.471863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:24.471863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:24.471863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:24.472696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:24.472696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:24.472696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:24.472696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:24.472696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:24.472696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:24.472696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:24.472696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:24.472696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:24.472696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:24.472696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:24.472696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:24.472696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:24.472696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:24.472696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:24.472696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:24.473881 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:24.473881 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:24.473881 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:24.473881 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:24.473881 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:24.473881 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:24.473881 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:24.473881 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:24.473881 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:24.473881 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:24.473881 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:15:24.473881 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:24.473881 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 124/260] 10.222.0.30 () {56 vars in 1135 bytes} [Wed Nov 2 07:15:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 39 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:24.478876 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323bb27d-1468-43e7-9b96-606370202964 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:24.489127 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323bb27d-1468-43e7-9b96-606370202964 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:24.493568 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1081d442-6e8d-4e37-af63-9380e1b6e2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:24.494188 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081d442-6e8d-4e37-af63-9380e1b6e2f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:24.494463 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081d442-6e8d-4e37-af63-9380e1b6e2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:24.494767 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081d442-6e8d-4e37-af63-9380e1b6e2f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:24.495554 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1081d442-6e8d-4e37-af63-9380e1b6e2f2 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:24.496095 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1081d442-6e8d-4e37-af63-9380e1b6e2f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:24.497584 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: WARNING py.warnings [None req-1081d442-6e8d-4e37-af63-9380e1b6e2f2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:15:24.497584 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: warnings.warn(msg) Nov 02 07:15:24.497584 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: Nov 02 07:15:24.498557 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1081d442-6e8d-4e37-af63-9380e1b6e2f2 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:24.507740 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 137/261] 10.222.0.30 () {56 vars in 1132 bytes} [Wed Nov 2 07:15:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:24.513421 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2802ef-9b45-4b55-8ea1-3ad931ea3ec7 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:24.523824 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2802ef-9b45-4b55-8ea1-3ad931ea3ec7 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:24.528615 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8c953-c05c-4c86-9eb3-52aa44f49a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:24.529604 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8c953-c05c-4c86-9eb3-52aa44f49a9d admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:24.529878 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8c953-c05c-4c86-9eb3-52aa44f49a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:24.530203 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8c953-c05c-4c86-9eb3-52aa44f49a9d admin admin] PATH_INFO: `/v3/groups/593c2c6ed3e64b14b3b10ebf8d8a75fe` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:24.537126 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd8c953-c05c-4c86-9eb3-52aa44f49a9d admin admin] RBAC: Authorizing `identity:get_group(group_id=593c2c6ed3e64b14b3b10ebf8d8a75fe)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:24.539014 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: WARNING py.warnings [None req-9cd8c953-c05c-4c86-9eb3-52aa44f49a9d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:15:24.539014 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: warnings.warn(msg) Nov 02 07:15:24.539014 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: Nov 02 07:15:24.540261 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd8c953-c05c-4c86-9eb3-52aa44f49a9d admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:24.542245 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 125/262] 10.222.0.30 () {56 vars in 1201 bytes} [Wed Nov 2 07:15:24 2022] GET /identity/v3/groups/593c2c6ed3e64b14b3b10ebf8d8a75fe => generated 226 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:24.547789 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1081d442-6e8d-4e37-af63-9380e1b6e2f2 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:24.559635 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1081d442-6e8d-4e37-af63-9380e1b6e2f2 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:24.564496 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ed851-3a08-4b16-9e3d-c5e838bdb8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:24.565166 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ed851-3a08-4b16-9e3d-c5e838bdb8e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:24.565442 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ed851-3a08-4b16-9e3d-c5e838bdb8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:24.565765 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ed851-3a08-4b16-9e3d-c5e838bdb8e1 admin admin] PATH_INFO: `/v3/projects/03fe633d52c14c2aa697f9116b41facc` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:24.566980 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ed851-3a08-4b16-9e3d-c5e838bdb8e1 admin admin] RBAC: Authorizing `identity:get_project(project_id=03fe633d52c14c2aa697f9116b41facc)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:24.569241 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ed851-3a08-4b16-9e3d-c5e838bdb8e1 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:24.571493 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 138/263] 10.222.0.30 () {56 vars in 1207 bytes} [Wed Nov 2 07:15:24 2022] GET /identity/v3/projects/03fe633d52c14c2aa697f9116b41facc => generated 298 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:24.577481 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8c953-c05c-4c86-9eb3-52aa44f49a9d admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:24.584411 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:24.589577 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8c953-c05c-4c86-9eb3-52aa44f49a9d admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:24.594362 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dec49e-5993-4d5d-8032-6d6c29826097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:24.595409 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dec49e-5993-4d5d-8032-6d6c29826097 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:24.595730 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dec49e-5993-4d5d-8032-6d6c29826097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:24.596195 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dec49e-5993-4d5d-8032-6d6c29826097 admin admin] PATH_INFO: `/v3/projects/03fe633d52c14c2aa697f9116b41facc/groups/593c2c6ed3e64b14b3b10ebf8d8a75fe/roles/39eab44f330440ebbefe39e538462165` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:24.605346 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dec49e-5993-4d5d-8032-6d6c29826097 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03fe633d52c14c2aa697f9116b41facc, group_id=593c2c6ed3e64b14b3b10ebf8d8a75fe, role_id=39eab44f330440ebbefe39e538462165)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:24.606054 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: WARNING py.warnings [None req-c8dec49e-5993-4d5d-8032-6d6c29826097 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:15:24.606054 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: warnings.warn(msg) Nov 02 07:15:24.606054 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: Nov 02 07:15:24.608389 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dec49e-5993-4d5d-8032-6d6c29826097 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:24.639884 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 126/264] 10.222.0.30 () {58 vars in 1463 bytes} [Wed Nov 2 07:15:24 2022] PUT /identity/v3/projects/03fe633d52c14c2aa697f9116b41facc/groups/593c2c6ed3e64b14b3b10ebf8d8a75fe/roles/39eab44f330440ebbefe39e538462165 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:24.753380 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6999bd-977f-40e2-a4de-95c2c757f697 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:24.753690 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6999bd-977f-40e2-a4de-95c2c757f697 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:24.753915 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6999bd-977f-40e2-a4de-95c2c757f697 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:24.755588 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 139/265] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:24.761879 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0961b-c6aa-4820-896f-204e6ec7f702 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:24.761879 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0961b-c6aa-4820-896f-204e6ec7f702 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:24.762274 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0961b-c6aa-4820-896f-204e6ec7f702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:24.788634 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-bfd0961b-c6aa-4820-896f-204e6ec7f702 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:24.789848 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-bfd0961b-c6aa-4820-896f-204e6ec7f702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:24.867773 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 127/266] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:24 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Nov 02 07:15:25.007848 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6999bd-977f-40e2-a4de-95c2c757f697 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:25.022209 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6999bd-977f-40e2-a4de-95c2c757f697 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:25.032582 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ee70f-b138-41db-8718-67f7473331c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:25.033220 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ee70f-b138-41db-8718-67f7473331c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:25.033492 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ee70f-b138-41db-8718-67f7473331c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:25.033760 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ee70f-b138-41db-8718-67f7473331c7 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:25.045116 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409ee70f-b138-41db-8718-67f7473331c7 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:25.050363 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-409ee70f-b138-41db-8718-67f7473331c7 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:25.050363 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:25.050363 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:25.050363 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:25.050363 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:25.050363 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:25.050363 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:25.050363 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:25.050363 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:25.050363 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:25.050363 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:25.050363 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:25.050363 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:25.050363 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:25.050363 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:25.050363 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:25.051232 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:25.051232 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:25.051232 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:25.051232 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:25.051232 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:25.051232 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:25.051232 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:25.051232 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:25.051232 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:25.051232 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:25.051232 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:25.051232 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:25.051232 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:25.051232 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:25.051232 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:25.051232 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:25.052536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:25.052536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:25.052536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:25.052536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:25.052536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:25.052536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:25.052536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:25.052536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:25.052536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:25.052536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:25.052536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:25.052536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:25.052536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 140/267] 10.222.0.30 () {56 vars in 1120 bytes} [Wed Nov 2 07:15:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 44 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:25.057346 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0961b-c6aa-4820-896f-204e6ec7f702 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:25.071809 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0961b-c6aa-4820-896f-204e6ec7f702 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:25.076430 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162db82-0cf5-4e1b-a3c0-a22124402e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:25.077046 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162db82-0cf5-4e1b-a3c0-a22124402e0e admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:25.077318 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162db82-0cf5-4e1b-a3c0-a22124402e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:25.077582 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162db82-0cf5-4e1b-a3c0-a22124402e0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:25.078249 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1162db82-0cf5-4e1b-a3c0-a22124402e0e admin admin] RBAC: Adding query filter params (name=member) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:25.078538 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1162db82-0cf5-4e1b-a3c0-a22124402e0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:25.079600 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: WARNING py.warnings [None req-1162db82-0cf5-4e1b-a3c0-a22124402e0e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:15:25.079600 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: warnings.warn(msg) Nov 02 07:15:25.079600 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: Nov 02 07:15:25.080477 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1162db82-0cf5-4e1b-a3c0-a22124402e0e admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:25.094839 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 128/268] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Nov 2 07:15:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:25.100210 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ee70f-b138-41db-8718-67f7473331c7 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:25.110581 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ee70f-b138-41db-8718-67f7473331c7 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:25.120132 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a2276-8e2a-4b5b-aee1-0c1c6b45e5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:25.120132 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a2276-8e2a-4b5b-aee1-0c1c6b45e5bc admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:25.120132 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a2276-8e2a-4b5b-aee1-0c1c6b45e5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:25.120132 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a2276-8e2a-4b5b-aee1-0c1c6b45e5bc admin admin] PATH_INFO: `/v3/groups/593c2c6ed3e64b14b3b10ebf8d8a75fe` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:25.120708 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609a2276-8e2a-4b5b-aee1-0c1c6b45e5bc admin admin] RBAC: Authorizing `identity:get_group(group_id=593c2c6ed3e64b14b3b10ebf8d8a75fe)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:25.120708 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: WARNING py.warnings [None req-609a2276-8e2a-4b5b-aee1-0c1c6b45e5bc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:15:25.120708 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: warnings.warn(msg) Nov 02 07:15:25.120708 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: Nov 02 07:15:25.120708 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609a2276-8e2a-4b5b-aee1-0c1c6b45e5bc admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:25.120708 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 141/269] 10.222.0.30 () {56 vars in 1201 bytes} [Wed Nov 2 07:15:25 2022] GET /identity/v3/groups/593c2c6ed3e64b14b3b10ebf8d8a75fe => generated 226 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:25.127049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162db82-0cf5-4e1b-a3c0-a22124402e0e admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:25.140425 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162db82-0cf5-4e1b-a3c0-a22124402e0e admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:25.147273 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf91023-8464-477f-abc2-e11917c7a77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:25.147919 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf91023-8464-477f-abc2-e11917c7a77e admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:25.148192 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf91023-8464-477f-abc2-e11917c7a77e admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:25.148514 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf91023-8464-477f-abc2-e11917c7a77e admin admin] PATH_INFO: `/v3/projects/03fe633d52c14c2aa697f9116b41facc` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:25.149616 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf91023-8464-477f-abc2-e11917c7a77e admin admin] RBAC: Authorizing `identity:get_project(project_id=03fe633d52c14c2aa697f9116b41facc)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:25.151759 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf91023-8464-477f-abc2-e11917c7a77e admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:25.153688 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 129/270] 10.222.0.30 () {56 vars in 1207 bytes} [Wed Nov 2 07:15:25 2022] GET /identity/v3/projects/03fe633d52c14c2aa697f9116b41facc => generated 298 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:25.159584 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a2276-8e2a-4b5b-aee1-0c1c6b45e5bc admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:25.172267 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a2276-8e2a-4b5b-aee1-0c1c6b45e5bc admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:25.176757 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23d588-b279-405c-9516-b6b35dc635e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:25.177423 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23d588-b279-405c-9516-b6b35dc635e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:25.177698 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23d588-b279-405c-9516-b6b35dc635e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:25.177964 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23d588-b279-405c-9516-b6b35dc635e9 admin admin] PATH_INFO: `/v3/projects/03fe633d52c14c2aa697f9116b41facc/groups/593c2c6ed3e64b14b3b10ebf8d8a75fe/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:25.183971 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23d588-b279-405c-9516-b6b35dc635e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03fe633d52c14c2aa697f9116b41facc, group_id=593c2c6ed3e64b14b3b10ebf8d8a75fe, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:25.185018 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: WARNING py.warnings [None req-6d23d588-b279-405c-9516-b6b35dc635e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:15:25.185018 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: warnings.warn(msg) Nov 02 07:15:25.185018 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: Nov 02 07:15:25.186982 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23d588-b279-405c-9516-b6b35dc635e9 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:25.217969 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 142/271] 10.222.0.30 () {58 vars in 1463 bytes} [Wed Nov 2 07:15:25 2022] PUT /identity/v3/projects/03fe633d52c14c2aa697f9116b41facc/groups/593c2c6ed3e64b14b3b10ebf8d8a75fe/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:25.282904 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf91023-8464-477f-abc2-e11917c7a77e admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:25.296882 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf91023-8464-477f-abc2-e11917c7a77e admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:25.371092 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1aabdf-22b4-48d6-b2b1-c78d6fde135e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:25.371838 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1aabdf-22b4-48d6-b2b1-c78d6fde135e admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:25.371999 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1aabdf-22b4-48d6-b2b1-c78d6fde135e admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:25.372268 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1aabdf-22b4-48d6-b2b1-c78d6fde135e admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:25.389022 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1aabdf-22b4-48d6-b2b1-c78d6fde135e admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:25.394475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-4e1aabdf-22b4-48d6-b2b1-c78d6fde135e admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:25.394475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:25.394475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:25.394475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:25.394475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:25.394475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:25.394475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:25.394475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:25.394475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:25.394475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:25.394475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:25.394475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:25.394475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:25.394475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:25.394475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:25.394475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:25.395421 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:25.395421 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:25.395421 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:25.395421 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:25.395421 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:25.395421 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:25.395421 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:25.395421 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:25.395421 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:25.395421 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:25.395421 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:25.395421 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:25.395421 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:25.395421 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:25.395421 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:25.395421 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:25.396635 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:25.396635 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:25.396635 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:25.396635 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:25.396635 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:25.396635 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:25.396635 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:25.396635 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:25.396635 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:25.396635 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:25.396635 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:25.396635 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:25.396635 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 130/272] 10.222.0.30 () {56 vars in 1120 bytes} [Wed Nov 2 07:15:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 114 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:25.404686 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23d588-b279-405c-9516-b6b35dc635e9 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:25.420823 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23d588-b279-405c-9516-b6b35dc635e9 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:25.425365 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfd04b-61c9-4df4-9a69-319e73275aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:25.425992 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfd04b-61c9-4df4-9a69-319e73275aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:25.426264 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfd04b-61c9-4df4-9a69-319e73275aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:25.426531 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfd04b-61c9-4df4-9a69-319e73275aa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:25.427242 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcfd04b-61c9-4df4-9a69-319e73275aa1 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:25.427538 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcfd04b-61c9-4df4-9a69-319e73275aa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:25.429190 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcfd04b-61c9-4df4-9a69-319e73275aa1 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:25.443909 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 143/273] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Nov 2 07:15:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:25.449903 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1aabdf-22b4-48d6-b2b1-c78d6fde135e admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:25.460345 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1aabdf-22b4-48d6-b2b1-c78d6fde135e admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:25.471190 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a237098-38d8-4390-8fda-826d65e6691d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:25.471856 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a237098-38d8-4390-8fda-826d65e6691d admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:25.472159 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a237098-38d8-4390-8fda-826d65e6691d admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:25.472480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a237098-38d8-4390-8fda-826d65e6691d admin admin] PATH_INFO: `/v3/groups/593c2c6ed3e64b14b3b10ebf8d8a75fe` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:25.473673 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a237098-38d8-4390-8fda-826d65e6691d admin admin] RBAC: Authorizing `identity:get_group(group_id=593c2c6ed3e64b14b3b10ebf8d8a75fe)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:25.475705 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a237098-38d8-4390-8fda-826d65e6691d admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:25.477668 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 131/274] 10.222.0.30 () {56 vars in 1201 bytes} [Wed Nov 2 07:15:25 2022] GET /identity/v3/groups/593c2c6ed3e64b14b3b10ebf8d8a75fe => generated 226 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:25.484229 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfd04b-61c9-4df4-9a69-319e73275aa1 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:25.500737 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfd04b-61c9-4df4-9a69-319e73275aa1 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:25.505297 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30cc7e-2a3a-4b45-a09c-3fa7960285c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:25.505944 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30cc7e-2a3a-4b45-a09c-3fa7960285c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:25.506211 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30cc7e-2a3a-4b45-a09c-3fa7960285c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:25.506479 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30cc7e-2a3a-4b45-a09c-3fa7960285c5 admin admin] PATH_INFO: `/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:25.511670 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b30cc7e-2a3a-4b45-a09c-3fa7960285c5 admin admin] RBAC: Authorizing `identity:get_project(project_id=ef9f41b2925a4825a22ae82a7bd2e955)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:25.513770 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b30cc7e-2a3a-4b45-a09c-3fa7960285c5 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:25.515800 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 144/275] 10.222.0.30 () {56 vars in 1207 bytes} [Wed Nov 2 07:15:25 2022] GET /identity/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955 => generated 302 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:25.521925 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a237098-38d8-4390-8fda-826d65e6691d admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:25.533196 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a237098-38d8-4390-8fda-826d65e6691d admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:25.543588 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714e1f6-0635-412e-87a2-861e6f2677c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:25.544289 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714e1f6-0635-412e-87a2-861e6f2677c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:25.544592 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714e1f6-0635-412e-87a2-861e6f2677c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:25.544900 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714e1f6-0635-412e-87a2-861e6f2677c8 admin admin] PATH_INFO: `/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955/groups/593c2c6ed3e64b14b3b10ebf8d8a75fe/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:25.546912 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a714e1f6-0635-412e-87a2-861e6f2677c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef9f41b2925a4825a22ae82a7bd2e955, group_id=593c2c6ed3e64b14b3b10ebf8d8a75fe, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:25.549587 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a714e1f6-0635-412e-87a2-861e6f2677c8 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:25.572829 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 132/276] 10.222.0.30 () {58 vars in 1463 bytes} [Wed Nov 2 07:15:25 2022] PUT /identity/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955/groups/593c2c6ed3e64b14b3b10ebf8d8a75fe/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:25.875247 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f030f9-e4d9-4df3-abad-5d7ec26d9eb1 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:25.875632 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f030f9-e4d9-4df3-abad-5d7ec26d9eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:25.875838 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f030f9-e4d9-4df3-abad-5d7ec26d9eb1 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:25.877127 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 145/277] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:25.883311 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc45a2-17a6-4134-ae3d-da69f80d75f9 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:25.883631 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc45a2-17a6-4134-ae3d-da69f80d75f9 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:25.883949 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc45a2-17a6-4134-ae3d-da69f80d75f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:25.905741 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-c1fc45a2-17a6-4134-ae3d-da69f80d75f9 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:25.907045 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-c1fc45a2-17a6-4134-ae3d-da69f80d75f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:25.966061 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:25.991548 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 133/278] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:15:26.027805 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79afb9-4e5f-4ad3-8f7a-69f63036c251 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:26.028327 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79afb9-4e5f-4ad3-8f7a-69f63036c251 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:26.028327 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79afb9-4e5f-4ad3-8f7a-69f63036c251 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:26.030002 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 146/279] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:26.036289 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9be7b-73a9-49da-b0d1-c16c99d4d3b2 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:26.036588 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9be7b-73a9-49da-b0d1-c16c99d4d3b2 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:26.036838 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9be7b-73a9-49da-b0d1-c16c99d4d3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:26.059010 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-13f9be7b-73a9-49da-b0d1-c16c99d4d3b2 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:26.060217 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-13f9be7b-73a9-49da-b0d1-c16c99d4d3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:26.127989 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 134/280] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:26 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Nov 02 07:15:26.484821 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79afb9-4e5f-4ad3-8f7a-69f63036c251 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:26.590549 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:26.619288 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79afb9-4e5f-4ad3-8f7a-69f63036c251 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:26.661127 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32373b9e-acd4-4b50-9d07-d07e192c1c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:26.661759 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32373b9e-acd4-4b50-9d07-d07e192c1c67 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:26.662030 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32373b9e-acd4-4b50-9d07-d07e192c1c67 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:26.662294 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32373b9e-acd4-4b50-9d07-d07e192c1c67 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:26.675864 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32373b9e-acd4-4b50-9d07-d07e192c1c67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:26.680630 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-32373b9e-acd4-4b50-9d07-d07e192c1c67 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:15:26.680630 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:26.680630 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:26.680630 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:26.680630 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:26.680630 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:26.680630 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:26.680630 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:26.680630 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:26.680630 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:26.680630 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:26.680630 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:26.680630 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:26.680630 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:26.680630 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:26.680630 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:26.681517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:26.681517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:26.681517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:26.681517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:26.681517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:26.681517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:26.681517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:26.681517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:26.681517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:26.681517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:26.681517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:26.681517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:26.681517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:26.681517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:26.681517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:26.681517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:26.682724 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:26.682724 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:26.682724 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:26.682724 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:26.682724 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:26.682724 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:26.682724 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:26.682724 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:26.682724 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:26.682724 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:26.682724 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 07:15:26.682724 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:26.682724 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 147/281] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:26.687585 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32373b9e-acd4-4b50-9d07-d07e192c1c67 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:26.687710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f9be7b-73a9-49da-b0d1-c16c99d4d3b2 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:26.699794 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32373b9e-acd4-4b50-9d07-d07e192c1c67 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:26.703959 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e3eb8-e433-40f3-9808-66b4cde8c076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:26.704586 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e3eb8-e433-40f3-9808-66b4cde8c076 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:26.704858 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e3eb8-e433-40f3-9808-66b4cde8c076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:26.705126 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e3eb8-e433-40f3-9808-66b4cde8c076 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:26.722285 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e3eb8-e433-40f3-9808-66b4cde8c076 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:26.727154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-073e3eb8-e433-40f3-9808-66b4cde8c076 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:15:26.727154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:26.727154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:26.727154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:26.727154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:26.727154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:26.727154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:26.727154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:26.727154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:26.727154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:26.727154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:26.727154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:26.727154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:26.727154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:26.727154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:26.727154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:26.728039 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:26.728039 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:26.728039 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:26.728039 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:26.728039 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:26.728039 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:26.728039 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:26.728039 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:26.728039 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:26.728039 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:26.728039 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:26.728039 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:26.728039 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:26.728039 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:26.728039 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:26.728039 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:26.729323 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:26.729323 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:26.729323 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:26.729323 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:26.729323 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:26.729323 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:26.729323 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:26.729323 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:26.729323 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:26.729323 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:26.729323 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:15:26.729323 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:26.729323 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 148/282] 10.222.0.30 () {56 vars in 1135 bytes} [Wed Nov 2 07:15:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 41 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:26.738760 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e3eb8-e433-40f3-9808-66b4cde8c076 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:26.766583 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e3eb8-e433-40f3-9808-66b4cde8c076 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:26.774689 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e44037-3cf1-4d6d-ae41-60d3199ea0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:26.775284 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e44037-3cf1-4d6d-ae41-60d3199ea0c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:26.775556 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e44037-3cf1-4d6d-ae41-60d3199ea0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:26.775824 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e44037-3cf1-4d6d-ae41-60d3199ea0c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:26.776492 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e44037-3cf1-4d6d-ae41-60d3199ea0c1 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:26.776826 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e44037-3cf1-4d6d-ae41-60d3199ea0c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:26.778615 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e44037-3cf1-4d6d-ae41-60d3199ea0c1 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:26.797147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 149/283] 10.222.0.30 () {56 vars in 1132 bytes} [Wed Nov 2 07:15:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:26.803797 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e44037-3cf1-4d6d-ae41-60d3199ea0c1 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:26.818518 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e44037-3cf1-4d6d-ae41-60d3199ea0c1 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:26.835209 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7403376-0a3f-4416-bb98-e9dbd83d3340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:26.835855 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7403376-0a3f-4416-bb98-e9dbd83d3340 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:26.836123 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7403376-0a3f-4416-bb98-e9dbd83d3340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:26.836389 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7403376-0a3f-4416-bb98-e9dbd83d3340 admin admin] PATH_INFO: `/v3/groups/593c2c6ed3e64b14b3b10ebf8d8a75fe` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:26.838649 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7403376-0a3f-4416-bb98-e9dbd83d3340 admin admin] RBAC: Authorizing `identity:get_group(group_id=593c2c6ed3e64b14b3b10ebf8d8a75fe)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:26.840635 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7403376-0a3f-4416-bb98-e9dbd83d3340 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:26.842260 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 150/284] 10.222.0.30 () {56 vars in 1201 bytes} [Wed Nov 2 07:15:26 2022] GET /identity/v3/groups/593c2c6ed3e64b14b3b10ebf8d8a75fe => generated 226 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:26.848476 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7403376-0a3f-4416-bb98-e9dbd83d3340 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:26.867038 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7403376-0a3f-4416-bb98-e9dbd83d3340 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:26.877622 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c980fa-6a72-4d97-94c3-8dabf4e31d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:26.878265 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c980fa-6a72-4d97-94c3-8dabf4e31d91 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:26.878536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c980fa-6a72-4d97-94c3-8dabf4e31d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:26.878842 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c980fa-6a72-4d97-94c3-8dabf4e31d91 admin admin] PATH_INFO: `/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:26.880641 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c980fa-6a72-4d97-94c3-8dabf4e31d91 admin admin] RBAC: Authorizing `identity:get_project(project_id=ef9f41b2925a4825a22ae82a7bd2e955)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:26.882769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c980fa-6a72-4d97-94c3-8dabf4e31d91 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:26.884440 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 151/285] 10.222.0.30 () {56 vars in 1207 bytes} [Wed Nov 2 07:15:26 2022] GET /identity/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955 => generated 302 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:26.892231 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f9be7b-73a9-49da-b0d1-c16c99d4d3b2 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:26.894606 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c980fa-6a72-4d97-94c3-8dabf4e31d91 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:26.910340 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c980fa-6a72-4d97-94c3-8dabf4e31d91 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:26.919325 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3312bca-cb79-443f-85b6-ac80016d5d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:26.920054 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3312bca-cb79-443f-85b6-ac80016d5d8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:26.920326 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3312bca-cb79-443f-85b6-ac80016d5d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:26.920596 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3312bca-cb79-443f-85b6-ac80016d5d8d admin admin] PATH_INFO: `/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955/groups/593c2c6ed3e64b14b3b10ebf8d8a75fe/roles/39eab44f330440ebbefe39e538462165` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:26.924200 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3312bca-cb79-443f-85b6-ac80016d5d8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef9f41b2925a4825a22ae82a7bd2e955, group_id=593c2c6ed3e64b14b3b10ebf8d8a75fe, role_id=39eab44f330440ebbefe39e538462165)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:26.927044 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3312bca-cb79-443f-85b6-ac80016d5d8d admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:26.940359 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:26.966688 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f5456-ffa7-4ad2-8b31-2d12359ea6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:26.967143 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 152/286] 10.222.0.30 () {58 vars in 1463 bytes} [Wed Nov 2 07:15:26 2022] PUT /identity/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955/groups/593c2c6ed3e64b14b3b10ebf8d8a75fe/roles/39eab44f330440ebbefe39e538462165 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:15:26.967389 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f5456-ffa7-4ad2-8b31-2d12359ea6f3 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:26.967666 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f5456-ffa7-4ad2-8b31-2d12359ea6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:26.967934 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f5456-ffa7-4ad2-8b31-2d12359ea6f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:26.968683 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f5456-ffa7-4ad2-8b31-2d12359ea6f3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:26.968984 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f5456-ffa7-4ad2-8b31-2d12359ea6f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:26.970413 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f5456-ffa7-4ad2-8b31-2d12359ea6f3 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:26.986553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 135/287] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 07:15:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:26.992202 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3312bca-cb79-443f-85b6-ac80016d5d8d admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:27.143342 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3312bca-cb79-443f-85b6-ac80016d5d8d admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:27.144862 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:27.188696 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b189dd-21e1-4a0c-a8da-3dc277da8ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:27.189408 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b189dd-21e1-4a0c-a8da-3dc277da8ba3 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:27.189684 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b189dd-21e1-4a0c-a8da-3dc277da8ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:27.190006 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b189dd-21e1-4a0c-a8da-3dc277da8ba3 None admin] PATH_INFO: `/v3/users/41a6cc02725648459037ee20691db41c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:27.192020 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b189dd-21e1-4a0c-a8da-3dc277da8ba3 None admin] RBAC: Authorizing `identity:get_user(user_id=41a6cc02725648459037ee20691db41c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:27.193484 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b189dd-21e1-4a0c-a8da-3dc277da8ba3 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:27.195513 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 153/288] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:15:26 2022] GET /identity/v3/users/41a6cc02725648459037ee20691db41c => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:27.201421 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f5456-ffa7-4ad2-8b31-2d12359ea6f3 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:27.320545 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f5456-ffa7-4ad2-8b31-2d12359ea6f3 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:27.365521 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0767ce8-436f-4170-a040-288973e9b39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:27.366557 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0767ce8-436f-4170-a040-288973e9b39a None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:27.367034 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0767ce8-436f-4170-a040-288973e9b39a None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:27.367471 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0767ce8-436f-4170-a040-288973e9b39a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:27.368384 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0767ce8-436f-4170-a040-288973e9b39a None admin] RBAC: Adding query filter params (role.id=0ca4aff56e484fa99225a67bae03722f, user.id=41a6cc02725648459037ee20691db41c, scope.system=all) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:27.368891 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0767ce8-436f-4170-a040-288973e9b39a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:27.370731 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0767ce8-436f-4170-a040-288973e9b39a None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:27.385188 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 136/289] 10.222.0.30 () {56 vars in 1308 bytes} [Wed Nov 2 07:15:27 2022] GET /identity/v3/role_assignments?user.id=41a6cc02725648459037ee20691db41c&scope.system=all&role.id=0ca4aff56e484fa99225a67bae03722f => generated 510 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:27.580806 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6bccb-d5d9-46e7-a6b6-232dfa2e4b47 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:27.581353 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6bccb-d5d9-46e7-a6b6-232dfa2e4b47 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:27.581826 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6bccb-d5d9-46e7-a6b6-232dfa2e4b47 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:27.583725 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 154/290] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:27.594517 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2bdeef-0923-4f07-9a18-2bbdd3574523 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:27.595211 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2bdeef-0923-4f07-9a18-2bbdd3574523 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:27.595673 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2bdeef-0923-4f07-9a18-2bbdd3574523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:27.620157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-1e2bdeef-0923-4f07-9a18-2bbdd3574523 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:27.621436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-1e2bdeef-0923-4f07-9a18-2bbdd3574523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:27.661093 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2badc24-ee90-4ae5-af64-47cf3b083e3e None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:27.661410 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2badc24-ee90-4ae5-af64-47cf3b083e3e None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:27.661639 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2badc24-ee90-4ae5-af64-47cf3b083e3e None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:27.663209 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 155/291] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:27.670595 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06db17b-aa64-4279-9f61-98bab701e8c7 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:27.670928 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06db17b-aa64-4279-9f61-98bab701e8c7 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:27.671168 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06db17b-aa64-4279-9f61-98bab701e8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:27.690807 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:27.704593 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-c06db17b-aa64-4279-9f61-98bab701e8c7 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:27.705796 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-c06db17b-aa64-4279-9f61-98bab701e8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:27.721590 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 137/292] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:27 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Nov 02 07:15:27.785757 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 156/293] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:15:28.170639 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2bdeef-0923-4f07-9a18-2bbdd3574523 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:28.179984 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2bdeef-0923-4f07-9a18-2bbdd3574523 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:28.186872 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5d0f1-cd15-4bb6-a9ac-23da72bbfaea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:28.187509 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5d0f1-cd15-4bb6-a9ac-23da72bbfaea admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:28.187830 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5d0f1-cd15-4bb6-a9ac-23da72bbfaea admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:28.188099 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5d0f1-cd15-4bb6-a9ac-23da72bbfaea admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:28.199174 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5d0f1-cd15-4bb6-a9ac-23da72bbfaea admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:28.204051 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-0be5d0f1-cd15-4bb6-a9ac-23da72bbfaea admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:28.204051 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:28.204051 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:28.204051 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:28.204051 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:28.204051 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:28.204051 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:28.204051 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:28.204051 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:28.204051 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:28.204051 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:28.204051 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:28.204051 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:28.204051 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:28.204051 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:28.204051 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:28.205348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:28.205348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:28.205348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:28.205348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:28.205348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:28.205348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:28.205348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:28.205348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:28.205348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:28.205348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:28.205348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:28.205348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:28.205348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:28.205348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:28.205348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:28.205348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:28.206124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:28.206124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:28.206124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:28.206124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:28.206124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:28.206124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:28.206124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:28.206124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:28.206124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:28.206124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:28.206124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:28.206124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:28.206124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 138/294] 10.222.0.30 () {56 vars in 1117 bytes} [Wed Nov 2 07:15:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 35 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:28.211117 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06db17b-aa64-4279-9f61-98bab701e8c7 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:28.222462 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06db17b-aa64-4279-9f61-98bab701e8c7 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:28.227034 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81250c-fa3e-4e5a-a7d3-df74df5db088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:28.227649 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81250c-fa3e-4e5a-a7d3-df74df5db088 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:28.227926 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81250c-fa3e-4e5a-a7d3-df74df5db088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:28.228189 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81250c-fa3e-4e5a-a7d3-df74df5db088 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:28.228851 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81250c-fa3e-4e5a-a7d3-df74df5db088 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:28.229149 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81250c-fa3e-4e5a-a7d3-df74df5db088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:28.230887 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81250c-fa3e-4e5a-a7d3-df74df5db088 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:28.240627 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 157/295] 10.222.0.30 () {56 vars in 1120 bytes} [Wed Nov 2 07:15:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:28.246297 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5d0f1-cd15-4bb6-a9ac-23da72bbfaea admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:28.257244 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5d0f1-cd15-4bb6-a9ac-23da72bbfaea admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:28.262683 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d558068d-4eb3-4b94-9c97-655ab13b11ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:28.263334 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558068d-4eb3-4b94-9c97-655ab13b11ca admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:28.263710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558068d-4eb3-4b94-9c97-655ab13b11ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:28.263975 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558068d-4eb3-4b94-9c97-655ab13b11ca admin admin] PATH_INFO: `/v3/groups/8ed292cec73f4645a120339bd68f80ec` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:28.269621 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d558068d-4eb3-4b94-9c97-655ab13b11ca admin admin] RBAC: Authorizing `identity:get_group(group_id=8ed292cec73f4645a120339bd68f80ec)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:28.271629 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d558068d-4eb3-4b94-9c97-655ab13b11ca admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:28.273226 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 139/296] 10.222.0.30 () {56 vars in 1201 bytes} [Wed Nov 2 07:15:28 2022] GET /identity/v3/groups/8ed292cec73f4645a120339bd68f80ec => generated 229 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:28.279088 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81250c-fa3e-4e5a-a7d3-df74df5db088 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:28.289843 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81250c-fa3e-4e5a-a7d3-df74df5db088 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:28.294597 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6eb360-ec00-4a61-8406-1a3db6a5d920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:28.295258 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6eb360-ec00-4a61-8406-1a3db6a5d920 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:28.295566 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6eb360-ec00-4a61-8406-1a3db6a5d920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:28.295839 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6eb360-ec00-4a61-8406-1a3db6a5d920 admin admin] PATH_INFO: `/v3/projects/b3f647c33aef4316a529d7730281cb6b` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:28.297056 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6eb360-ec00-4a61-8406-1a3db6a5d920 admin admin] RBAC: Authorizing `identity:get_project(project_id=b3f647c33aef4316a529d7730281cb6b)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:28.298997 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6eb360-ec00-4a61-8406-1a3db6a5d920 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:28.301022 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 158/297] 10.222.0.30 () {56 vars in 1207 bytes} [Wed Nov 2 07:15:28 2022] GET /identity/v3/projects/b3f647c33aef4316a529d7730281cb6b => generated 344 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:28.306742 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d558068d-4eb3-4b94-9c97-655ab13b11ca admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:28.316933 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6eb360-ec00-4a61-8406-1a3db6a5d920 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:28.317583 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d558068d-4eb3-4b94-9c97-655ab13b11ca admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:28.324585 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736cff7b-fee1-4580-9b5e-c308cf6640f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:28.325305 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736cff7b-fee1-4580-9b5e-c308cf6640f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:28.325578 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736cff7b-fee1-4580-9b5e-c308cf6640f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:28.325891 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736cff7b-fee1-4580-9b5e-c308cf6640f1 admin admin] PATH_INFO: `/v3/projects/b3f647c33aef4316a529d7730281cb6b/groups/8ed292cec73f4645a120339bd68f80ec/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:28.327948 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736cff7b-fee1-4580-9b5e-c308cf6640f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3f647c33aef4316a529d7730281cb6b, group_id=8ed292cec73f4645a120339bd68f80ec, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:28.333161 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736cff7b-fee1-4580-9b5e-c308cf6640f1 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:28.354486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 140/298] 10.222.0.30 () {58 vars in 1463 bytes} [Wed Nov 2 07:15:28 2022] PUT /identity/v3/projects/b3f647c33aef4316a529d7730281cb6b/groups/8ed292cec73f4645a120339bd68f80ec/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:28.468072 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:28.486847 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6eb360-ec00-4a61-8406-1a3db6a5d920 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:28.535871 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f838c25-71fa-4cc2-83ee-63f9b53d1c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:28.536536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f838c25-71fa-4cc2-83ee-63f9b53d1c37 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:28.536816 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f838c25-71fa-4cc2-83ee-63f9b53d1c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:28.537087 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f838c25-71fa-4cc2-83ee-63f9b53d1c37 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:28.555719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f838c25-71fa-4cc2-83ee-63f9b53d1c37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:28.561446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-7f838c25-71fa-4cc2-83ee-63f9b53d1c37 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:15:28.561446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:28.561446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:28.561446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:28.561446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:28.561446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:28.561446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:28.561446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:28.561446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:28.561446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:28.561446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:28.561446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:28.561446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:28.561446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:28.561446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:28.561446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:28.562388 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:28.562388 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:28.562388 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:28.562388 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:28.562388 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:28.562388 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:28.562388 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:28.562388 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:28.562388 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:28.562388 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:28.562388 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:28.562388 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:28.562388 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:28.562388 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:28.562388 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:28.562388 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:28.563201 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:28.563201 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:28.563201 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:28.563201 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:28.563201 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:28.563201 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:28.563201 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:28.563201 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:28.563201 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:28.563201 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:28.563201 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:15:28.563201 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:28.563201 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 159/299] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Nov 2 07:15:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:28.569050 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736cff7b-fee1-4580-9b5e-c308cf6640f1 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:28.713045 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736cff7b-fee1-4580-9b5e-c308cf6640f1 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:28.753754 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affbb2ab-1d1a-46f9-be7d-2a1d3d50581c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:28.754803 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affbb2ab-1d1a-46f9-be7d-2a1d3d50581c None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:28.755265 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affbb2ab-1d1a-46f9-be7d-2a1d3d50581c None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:28.755779 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affbb2ab-1d1a-46f9-be7d-2a1d3d50581c None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:28.756676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affbb2ab-1d1a-46f9-be7d-2a1d3d50581c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:28.757242 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affbb2ab-1d1a-46f9-be7d-2a1d3d50581c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:28.758872 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affbb2ab-1d1a-46f9-be7d-2a1d3d50581c None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:28.768896 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 141/300] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:28.774960 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f838c25-71fa-4cc2-83ee-63f9b53d1c37 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:28.821328 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5efb4e8-2d5a-4af5-8111-8571a17b6710 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:28.821930 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5efb4e8-2d5a-4af5-8111-8571a17b6710 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:28.822796 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5efb4e8-2d5a-4af5-8111-8571a17b6710 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:28.825427 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 142/301] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:15:28.833924 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389a5e4-72c3-4b54-be0a-404d4ca9ae81 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:28.834829 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389a5e4-72c3-4b54-be0a-404d4ca9ae81 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:28.835315 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389a5e4-72c3-4b54-be0a-404d4ca9ae81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:28.867680 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-7389a5e4-72c3-4b54-be0a-404d4ca9ae81 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:28.869024 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-7389a5e4-72c3-4b54-be0a-404d4ca9ae81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:28.886047 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:28.898075 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f838c25-71fa-4cc2-83ee-63f9b53d1c37 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:28.936940 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3958905d-6879-4a25-86a4-66436892ac14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:28.938573 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958905d-6879-4a25-86a4-66436892ac14 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:28.939153 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958905d-6879-4a25-86a4-66436892ac14 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:28.939637 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958905d-6879-4a25-86a4-66436892ac14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:28.945696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 143/302] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:28.951215 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb1c32a-d440-4594-bb01-b97a5860ba5a None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:28.951474 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb1c32a-d440-4594-bb01-b97a5860ba5a None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:28.951751 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb1c32a-d440-4594-bb01-b97a5860ba5a None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:28.954687 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 144/303] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:28.960362 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:28.964983 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde90d4-4be3-4ccf-978e-27c9b7f134fb None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:28.965264 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde90d4-4be3-4ccf-978e-27c9b7f134fb None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:28.965507 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde90d4-4be3-4ccf-978e-27c9b7f134fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:29.012307 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-2dde90d4-4be3-4ccf-978e-27c9b7f134fb None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:29.015109 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-2dde90d4-4be3-4ccf-978e-27c9b7f134fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:29.044825 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3958905d-6879-4a25-86a4-66436892ac14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:29.046369 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3958905d-6879-4a25-86a4-66436892ac14 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:29.094833 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 145/304] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:15:29.155633 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 160/305] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:15:29.161283 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dde90d4-4be3-4ccf-978e-27c9b7f134fb None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:29.255753 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:29.277128 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dde90d4-4be3-4ccf-978e-27c9b7f134fb None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:29.314568 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d581b348-626a-47ed-9e89-7a2fc2ae58dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:29.315606 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581b348-626a-47ed-9e89-7a2fc2ae58dd None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:29.316093 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581b348-626a-47ed-9e89-7a2fc2ae58dd None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:29.316550 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581b348-626a-47ed-9e89-7a2fc2ae58dd None admin] PATH_INFO: `/v3/projects/03fe633d52c14c2aa697f9116b41facc` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:29.317877 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d581b348-626a-47ed-9e89-7a2fc2ae58dd None admin] RBAC: Authorizing `identity:get_project(project_id=03fe633d52c14c2aa697f9116b41facc)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:29.319553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d581b348-626a-47ed-9e89-7a2fc2ae58dd None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:29.321739 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 146/306] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 07:15:29 2022] GET /identity/v3/projects/03fe633d52c14c2aa697f9116b41facc => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:29.327209 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3958905d-6879-4a25-86a4-66436892ac14 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:29.336779 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d581b348-626a-47ed-9e89-7a2fc2ae58dd None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:29.390309 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:29.456746 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3958905d-6879-4a25-86a4-66436892ac14 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:29.522634 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac247869-8250-4448-b52c-ef24864b2148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:29.523324 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac247869-8250-4448-b52c-ef24864b2148 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:29.523595 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac247869-8250-4448-b52c-ef24864b2148 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:29.523859 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac247869-8250-4448-b52c-ef24864b2148 None admin] PATH_INFO: `/v3/groups/593c2c6ed3e64b14b3b10ebf8d8a75fe` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:29.525028 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac247869-8250-4448-b52c-ef24864b2148 None admin] RBAC: Authorizing `identity:get_group(group_id=593c2c6ed3e64b14b3b10ebf8d8a75fe)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:29.526460 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac247869-8250-4448-b52c-ef24864b2148 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:29.528256 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 161/307] 10.222.0.30 () {56 vars in 1180 bytes} [Wed Nov 2 07:15:29 2022] GET /identity/v3/groups/593c2c6ed3e64b14b3b10ebf8d8a75fe => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:29.529263 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac247869-8250-4448-b52c-ef24864b2148 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:29.551513 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d581b348-626a-47ed-9e89-7a2fc2ae58dd None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:29.602853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd749bae-e3e8-48f0-bfa6-2fe80d29fb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:29.603768 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd749bae-e3e8-48f0-bfa6-2fe80d29fb02 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:29.603768 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd749bae-e3e8-48f0-bfa6-2fe80d29fb02 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:29.604014 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd749bae-e3e8-48f0-bfa6-2fe80d29fb02 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:29.621712 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd749bae-e3e8-48f0-bfa6-2fe80d29fb02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:29.627337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-fd749bae-e3e8-48f0-bfa6-2fe80d29fb02 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:29.627337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:29.627337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:29.627337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:29.627337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:29.627337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:29.627337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:29.627337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:29.627337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:29.627337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:29.627337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:29.627337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:29.627337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:29.627337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:29.627337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:29.627337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:29.628265 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:29.628265 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:29.628265 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:29.628265 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:29.628265 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:29.628265 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:29.628265 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:29.628265 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:29.628265 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:29.628265 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:29.628265 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:29.628265 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:29.628265 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:29.628265 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:29.628265 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:29.628265 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:29.629653 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:29.629653 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:29.629653 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:29.629653 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:29.629653 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:29.629653 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:29.629653 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:29.629653 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:29.629653 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:29.629653 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:29.629653 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:29.629653 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:29.629653 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 147/308] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:29.629653 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd749bae-e3e8-48f0-bfa6-2fe80d29fb02 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:29.656486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:29.676755 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac247869-8250-4448-b52c-ef24864b2148 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:29.694713 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:29.735167 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c64d0a0-9ed7-44b4-8646-e67c7e3dac0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:29.735810 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c64d0a0-9ed7-44b4-8646-e67c7e3dac0a None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:29.736086 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c64d0a0-9ed7-44b4-8646-e67c7e3dac0a None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:29.736354 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c64d0a0-9ed7-44b4-8646-e67c7e3dac0a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:29.748488 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c64d0a0-9ed7-44b4-8646-e67c7e3dac0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:29.753796 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-9c64d0a0-9ed7-44b4-8646-e67c7e3dac0a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:29.753796 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:29.753796 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:29.753796 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:29.753796 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:29.753796 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:29.753796 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:29.753796 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:29.753796 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:29.753796 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:29.753796 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:29.753796 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:29.753796 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:29.753796 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:29.753796 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:29.753796 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:29.754921 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:29.754921 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:29.754921 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:29.754921 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:29.754921 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:29.754921 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:29.754921 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:29.754921 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:29.754921 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:29.754921 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:29.754921 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:29.754921 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:29.754921 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:29.754921 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:29.754921 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:29.754921 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:29.755902 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:29.755902 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:29.755902 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:29.755902 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:29.755902 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:29.755902 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:29.755902 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:29.755902 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:29.755902 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:29.755902 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:29.755902 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 07:15:29.755902 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:29.755902 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 162/309] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:29.755902 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c64d0a0-9ed7-44b4-8646-e67c7e3dac0a None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:29.768812 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd749bae-e3e8-48f0-bfa6-2fe80d29fb02 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:29.808639 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b5e93-1ba2-479d-9671-56c6b94f55e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:29.809348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b5e93-1ba2-479d-9671-56c6b94f55e4 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:29.809642 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b5e93-1ba2-479d-9671-56c6b94f55e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:29.809912 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b5e93-1ba2-479d-9671-56c6b94f55e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:29.810647 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9b5e93-1ba2-479d-9671-56c6b94f55e4 None admin] RBAC: Adding query filter params (group.id=593c2c6ed3e64b14b3b10ebf8d8a75fe, role.id=39eab44f330440ebbefe39e538462165, scope.project.id=03fe633d52c14c2aa697f9116b41facc) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:29.810977 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9b5e93-1ba2-479d-9671-56c6b94f55e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:29.812533 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9b5e93-1ba2-479d-9671-56c6b94f55e4 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:29.820303 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 148/310] 10.222.0.30 () {56 vars in 1376 bytes} [Wed Nov 2 07:15:29 2022] GET /identity/v3/role_assignments?group.id=593c2c6ed3e64b14b3b10ebf8d8a75fe&scope.project.id=03fe633d52c14c2aa697f9116b41facc&role.id=39eab44f330440ebbefe39e538462165 => generated 611 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:29.821133 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b5e93-1ba2-479d-9671-56c6b94f55e4 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:29.918221 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c64d0a0-9ed7-44b4-8646-e67c7e3dac0a None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:29.980034 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:29.984497 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bacd6cb-dfa3-4152-ac0b-7d6b26972898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:29.985621 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bacd6cb-dfa3-4152-ac0b-7d6b26972898 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:29.986113 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bacd6cb-dfa3-4152-ac0b-7d6b26972898 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:29.987506 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bacd6cb-dfa3-4152-ac0b-7d6b26972898 None admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:29.991226 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bacd6cb-dfa3-4152-ac0b-7d6b26972898 None admin] RBAC: Adding query filter params (name=member) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:29.991226 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bacd6cb-dfa3-4152-ac0b-7d6b26972898 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:29.991226 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bacd6cb-dfa3-4152-ac0b-7d6b26972898 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:30.003515 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 163/311] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 07:15:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:30.009963 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bacd6cb-dfa3-4152-ac0b-7d6b26972898 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:30.051884 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b5e93-1ba2-479d-9671-56c6b94f55e4 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:30.072754 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:30.105135 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3bee9-2cc5-4fc2-a5a4-28d09d676d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:30.105793 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3bee9-2cc5-4fc2-a5a4-28d09d676d82 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:30.106075 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3bee9-2cc5-4fc2-a5a4-28d09d676d82 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:30.106345 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3bee9-2cc5-4fc2-a5a4-28d09d676d82 None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:30.107075 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af3bee9-2cc5-4fc2-a5a4-28d09d676d82 None admin] RBAC: Adding query filter params (name=member) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:30.107376 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af3bee9-2cc5-4fc2-a5a4-28d09d676d82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:30.108861 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af3bee9-2cc5-4fc2-a5a4-28d09d676d82 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:30.119759 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 149/312] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 07:15:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:30.125507 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3bee9-2cc5-4fc2-a5a4-28d09d676d82 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:30.149587 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bacd6cb-dfa3-4152-ac0b-7d6b26972898 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:30.196471 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f8cba-18ed-4d91-85c3-e8db4025a50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:30.197108 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f8cba-18ed-4d91-85c3-e8db4025a50c None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:30.197388 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f8cba-18ed-4d91-85c3-e8db4025a50c None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:30.197658 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f8cba-18ed-4d91-85c3-e8db4025a50c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:30.245682 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3bee9-2cc5-4fc2-a5a4-28d09d676d82 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:30.272496 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100f8cba-18ed-4d91-85c3-e8db4025a50c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:30.273990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100f8cba-18ed-4d91-85c3-e8db4025a50c None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:30.283266 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c786a9f4-1b68-4277-8c67-1a290b769315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:30.283905 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c786a9f4-1b68-4277-8c67-1a290b769315 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:30.284193 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c786a9f4-1b68-4277-8c67-1a290b769315 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:30.284463 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c786a9f4-1b68-4277-8c67-1a290b769315 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:30.288170 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:30.315126 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:30.365287 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c786a9f4-1b68-4277-8c67-1a290b769315 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:30.366790 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c786a9f4-1b68-4277-8c67-1a290b769315 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:30.391881 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 164/313] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Nov 02 07:15:30.393176 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63921fd8-c7a8-48a6-bee7-52f807050c24 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:30.393453 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63921fd8-c7a8-48a6-bee7-52f807050c24 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:30.393720 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63921fd8-c7a8-48a6-bee7-52f807050c24 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:30.399460 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 165/314] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:30.400982 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63921fd8-c7a8-48a6-bee7-52f807050c24 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:30.504914 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 150/315] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:15:30.506201 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a984b-6448-4462-8f56-5ba07bf09502 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:30.506987 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a984b-6448-4462-8f56-5ba07bf09502 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:30.507268 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a984b-6448-4462-8f56-5ba07bf09502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:30.526071 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-b84a984b-6448-4462-8f56-5ba07bf09502 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:30.527033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-b84a984b-6448-4462-8f56-5ba07bf09502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:30.527162 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63921fd8-c7a8-48a6-bee7-52f807050c24 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:30.559306 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:30.576648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a815029-7b66-420c-9b32-3ce8cab958ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:30.577651 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a815029-7b66-420c-9b32-3ce8cab958ae None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:30.577998 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a815029-7b66-420c-9b32-3ce8cab958ae None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:30.578366 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a815029-7b66-420c-9b32-3ce8cab958ae None admin] PATH_INFO: `/v3/projects/03fe633d52c14c2aa697f9116b41facc` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:30.579985 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a815029-7b66-420c-9b32-3ce8cab958ae None admin] RBAC: Authorizing `identity:get_project(project_id=03fe633d52c14c2aa697f9116b41facc)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:30.581752 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a815029-7b66-420c-9b32-3ce8cab958ae None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:30.583661 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 166/316] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 07:15:30 2022] GET /identity/v3/projects/03fe633d52c14c2aa697f9116b41facc => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:30.584237 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a815029-7b66-420c-9b32-3ce8cab958ae None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:30.587814 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:30.615809 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 151/317] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:30.623634 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a984b-6448-4462-8f56-5ba07bf09502 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:30.699714 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a815029-7b66-420c-9b32-3ce8cab958ae None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:30.741598 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2a025-4fa4-404c-9e35-ef1d94efc5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:30.742300 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2a025-4fa4-404c-9e35-ef1d94efc5c2 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:30.742693 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2a025-4fa4-404c-9e35-ef1d94efc5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:30.743080 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2a025-4fa4-404c-9e35-ef1d94efc5c2 None admin] PATH_INFO: `/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:30.744306 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a2a025-4fa4-404c-9e35-ef1d94efc5c2 None admin] RBAC: Authorizing `identity:get_project(project_id=ef9f41b2925a4825a22ae82a7bd2e955)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:30.745879 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a2a025-4fa4-404c-9e35-ef1d94efc5c2 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:30.748257 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 167/318] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 07:15:30 2022] GET /identity/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:30.749325 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a984b-6448-4462-8f56-5ba07bf09502 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:30.753941 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2a025-4fa4-404c-9e35-ef1d94efc5c2 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:30.794645 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c68275-2cfd-4df9-b47b-47c5f7213387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:30.795575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c68275-2cfd-4df9-b47b-47c5f7213387 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:30.796768 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c68275-2cfd-4df9-b47b-47c5f7213387 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:30.797050 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c68275-2cfd-4df9-b47b-47c5f7213387 None admin] PATH_INFO: `/v3/groups/593c2c6ed3e64b14b3b10ebf8d8a75fe` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:30.799546 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c68275-2cfd-4df9-b47b-47c5f7213387 None admin] RBAC: Authorizing `identity:get_group(group_id=593c2c6ed3e64b14b3b10ebf8d8a75fe)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:30.801051 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c68275-2cfd-4df9-b47b-47c5f7213387 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:30.803337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 152/319] 10.222.0.30 () {56 vars in 1180 bytes} [Wed Nov 2 07:15:30 2022] GET /identity/v3/groups/593c2c6ed3e64b14b3b10ebf8d8a75fe => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:30.809847 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c68275-2cfd-4df9-b47b-47c5f7213387 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:30.839023 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:30.851180 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:30.877538 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2a025-4fa4-404c-9e35-ef1d94efc5c2 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:30.916371 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b1a6e-3d72-43bb-9d51-4cfdab8b026a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:30.917012 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b1a6e-3d72-43bb-9d51-4cfdab8b026a None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:30.917387 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b1a6e-3d72-43bb-9d51-4cfdab8b026a None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:30.917596 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b1a6e-3d72-43bb-9d51-4cfdab8b026a None admin] PATH_INFO: `/v3/groups/593c2c6ed3e64b14b3b10ebf8d8a75fe` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:30.918867 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b1a6e-3d72-43bb-9d51-4cfdab8b026a None admin] RBAC: Authorizing `identity:get_group(group_id=593c2c6ed3e64b14b3b10ebf8d8a75fe)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:30.920441 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b1a6e-3d72-43bb-9d51-4cfdab8b026a None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:30.922199 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 168/320] 10.222.0.30 () {56 vars in 1180 bytes} [Wed Nov 2 07:15:30 2022] GET /identity/v3/groups/593c2c6ed3e64b14b3b10ebf8d8a75fe => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:30.924436 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b1a6e-3d72-43bb-9d51-4cfdab8b026a None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:30.940002 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c68275-2cfd-4df9-b47b-47c5f7213387 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:30.982232 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f35b5-858f-4aa8-b9e0-f4b388d293a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:30.982972 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f35b5-858f-4aa8-b9e0-f4b388d293a5 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:30.983233 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f35b5-858f-4aa8-b9e0-f4b388d293a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:30.983511 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f35b5-858f-4aa8-b9e0-f4b388d293a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:30.984241 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f35b5-858f-4aa8-b9e0-f4b388d293a5 None admin] RBAC: Adding query filter params (group.id=593c2c6ed3e64b14b3b10ebf8d8a75fe, role.id=15930dae1c0b464f98680edb03b7f02c, scope.project.id=03fe633d52c14c2aa697f9116b41facc) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:30.984546 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f35b5-858f-4aa8-b9e0-f4b388d293a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:30.985987 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f35b5-858f-4aa8-b9e0-f4b388d293a5 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:30.992920 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 153/321] 10.222.0.30 () {56 vars in 1376 bytes} [Wed Nov 2 07:15:30 2022] GET /identity/v3/role_assignments?group.id=593c2c6ed3e64b14b3b10ebf8d8a75fe&scope.project.id=03fe633d52c14c2aa697f9116b41facc&role.id=15930dae1c0b464f98680edb03b7f02c => generated 611 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:30.993755 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f35b5-858f-4aa8-b9e0-f4b388d293a5 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:31.052683 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b1a6e-3d72-43bb-9d51-4cfdab8b026a None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:31.094068 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3745ecec-9525-477f-bfa3-cb5914f25a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:31.094755 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745ecec-9525-477f-bfa3-cb5914f25a41 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:31.095037 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745ecec-9525-477f-bfa3-cb5914f25a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:31.095314 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745ecec-9525-477f-bfa3-cb5914f25a41 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:31.104285 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:31.110947 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3745ecec-9525-477f-bfa3-cb5914f25a41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:31.116376 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-3745ecec-9525-477f-bfa3-cb5914f25a41 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:15:31.116376 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:31.116376 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:31.116376 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:31.116376 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:31.116376 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:31.116376 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:31.116376 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:31.116376 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:31.116376 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:31.116376 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:31.116376 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:31.116376 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:31.116376 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:31.116376 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:31.116376 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:31.117303 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:31.117303 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:31.117303 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:31.117303 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:31.117303 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:31.117303 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:31.117303 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:31.117303 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:31.117303 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:31.117303 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:31.117303 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:31.117303 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:31.117303 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:31.117303 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:31.117303 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:31.117303 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:31.118107 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:31.118107 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:31.118107 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:31.118107 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:31.118107 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:31.118107 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:31.118107 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:31.118107 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:31.118107 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:31.118107 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:31.118107 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:15:31.118107 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:31.118107 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 169/322] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Nov 2 07:15:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:31.125593 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3745ecec-9525-477f-bfa3-cb5914f25a41 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:31.132323 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:31.141199 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f35b5-858f-4aa8-b9e0-f4b388d293a5 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:31.192465 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744bcd5-a8a1-44e6-9a60-15094121d692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:31.192952 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744bcd5-a8a1-44e6-9a60-15094121d692 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:31.193226 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744bcd5-a8a1-44e6-9a60-15094121d692 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:31.193666 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744bcd5-a8a1-44e6-9a60-15094121d692 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:31.194694 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3744bcd5-a8a1-44e6-9a60-15094121d692 None admin] RBAC: Adding query filter params (group.id=593c2c6ed3e64b14b3b10ebf8d8a75fe, role.id=15930dae1c0b464f98680edb03b7f02c, scope.project.id=ef9f41b2925a4825a22ae82a7bd2e955) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:31.194869 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3744bcd5-a8a1-44e6-9a60-15094121d692 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:31.196341 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3744bcd5-a8a1-44e6-9a60-15094121d692 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:31.205788 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 154/323] 10.222.0.30 () {56 vars in 1376 bytes} [Wed Nov 2 07:15:30 2022] GET /identity/v3/role_assignments?group.id=593c2c6ed3e64b14b3b10ebf8d8a75fe&scope.project.id=ef9f41b2925a4825a22ae82a7bd2e955&role.id=15930dae1c0b464f98680edb03b7f02c => generated 611 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:31.207167 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4cfb2-2c63-4e73-b8b5-537db13c4ed3 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:31.207457 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4cfb2-2c63-4e73-b8b5-537db13c4ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:31.207726 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4cfb2-2c63-4e73-b8b5-537db13c4ed3 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:31.216811 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 155/324] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:31.216811 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333a5f9-5ab1-462a-9353-5b723945e2c8 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:31.216811 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333a5f9-5ab1-462a-9353-5b723945e2c8 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:31.216811 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333a5f9-5ab1-462a-9353-5b723945e2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:31.238871 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-b333a5f9-5ab1-462a-9353-5b723945e2c8 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:31.240552 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-b333a5f9-5ab1-462a-9353-5b723945e2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:31.267715 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3745ecec-9525-477f-bfa3-cb5914f25a41 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:31.315256 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b515fea8-02ff-46ea-a4bf-aa9a1e60451e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:31.315888 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b515fea8-02ff-46ea-a4bf-aa9a1e60451e None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:31.316161 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b515fea8-02ff-46ea-a4bf-aa9a1e60451e None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:31.316430 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b515fea8-02ff-46ea-a4bf-aa9a1e60451e None admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:31.317109 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b515fea8-02ff-46ea-a4bf-aa9a1e60451e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:31.317411 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b515fea8-02ff-46ea-a4bf-aa9a1e60451e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:31.318854 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b515fea8-02ff-46ea-a4bf-aa9a1e60451e None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:31.330004 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 170/325] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:31.335889 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b515fea8-02ff-46ea-a4bf-aa9a1e60451e None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:31.340209 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 156/326] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:31.409261 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:31.455531 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b515fea8-02ff-46ea-a4bf-aa9a1e60451e None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:31.491210 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2227002-18d9-4469-ba8c-d0422c98c215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:31.492211 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2227002-18d9-4469-ba8c-d0422c98c215 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:31.492684 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2227002-18d9-4469-ba8c-d0422c98c215 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:31.493145 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2227002-18d9-4469-ba8c-d0422c98c215 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:31.566270 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2227002-18d9-4469-ba8c-d0422c98c215 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:31.567827 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2227002-18d9-4469-ba8c-d0422c98c215 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:31.631302 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333a5f9-5ab1-462a-9353-5b723945e2c8 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:31.639397 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:31.678324 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 171/327] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Nov 02 07:15:31.685197 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2227002-18d9-4469-ba8c-d0422c98c215 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:31.741382 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:31.756270 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333a5f9-5ab1-462a-9353-5b723945e2c8 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:31.794618 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2227002-18d9-4469-ba8c-d0422c98c215 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:31.799311 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342cf8c-3a5c-4426-ac33-1a923d5eb668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:31.799939 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342cf8c-3a5c-4426-ac33-1a923d5eb668 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:31.800214 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342cf8c-3a5c-4426-ac33-1a923d5eb668 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:31.800480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342cf8c-3a5c-4426-ac33-1a923d5eb668 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:31.816917 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8342cf8c-3a5c-4426-ac33-1a923d5eb668 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:31.821778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-8342cf8c-3a5c-4426-ac33-1a923d5eb668 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:31.821778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:31.821778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:31.821778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:31.821778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:31.821778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:31.821778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:31.821778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:31.821778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:31.821778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:31.821778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:31.821778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:31.821778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:31.821778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:31.821778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:31.821778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:31.822705 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:31.822705 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:31.822705 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:31.822705 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:31.822705 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:31.822705 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:31.822705 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:31.822705 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:31.822705 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:31.822705 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:31.822705 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:31.822705 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:31.822705 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:31.822705 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:31.822705 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:31.822705 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:31.823510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:31.823510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:31.823510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:31.823510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:31.823510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:31.823510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:31.823510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:31.823510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:31.823510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:31.823510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:31.823510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:31.823510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:31.824682 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 157/328] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 07:15:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:31.830205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342cf8c-3a5c-4426-ac33-1a923d5eb668 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:31.837331 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49aceb-70ac-4fca-825f-91d8be521bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:31.837971 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49aceb-70ac-4fca-825f-91d8be521bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:31.838245 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49aceb-70ac-4fca-825f-91d8be521bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:31.841687 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49aceb-70ac-4fca-825f-91d8be521bc6 None admin] PATH_INFO: `/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:31.841687 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49aceb-70ac-4fca-825f-91d8be521bc6 None admin] RBAC: Authorizing `identity:get_project(project_id=ef9f41b2925a4825a22ae82a7bd2e955)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:31.841687 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49aceb-70ac-4fca-825f-91d8be521bc6 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:31.843402 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 172/329] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 07:15:31 2022] GET /identity/v3/projects/ef9f41b2925a4825a22ae82a7bd2e955 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:31.848695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49aceb-70ac-4fca-825f-91d8be521bc6 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:31.881261 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:31.953905 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342cf8c-3a5c-4426-ac33-1a923d5eb668 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:31.961200 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49aceb-70ac-4fca-825f-91d8be521bc6 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:31.997819 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46aafef-59cb-4d57-b43f-3fdd488be3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:31.998466 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46aafef-59cb-4d57-b43f-3fdd488be3f5 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:31.998771 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46aafef-59cb-4d57-b43f-3fdd488be3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:31.999072 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46aafef-59cb-4d57-b43f-3fdd488be3f5 None admin] PATH_INFO: `/v3/groups/593c2c6ed3e64b14b3b10ebf8d8a75fe` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:32.000153 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46aafef-59cb-4d57-b43f-3fdd488be3f5 None admin] RBAC: Authorizing `identity:get_group(group_id=593c2c6ed3e64b14b3b10ebf8d8a75fe)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:32.001600 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46aafef-59cb-4d57-b43f-3fdd488be3f5 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:32.003235 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 173/330] 10.222.0.30 () {56 vars in 1180 bytes} [Wed Nov 2 07:15:31 2022] GET /identity/v3/groups/593c2c6ed3e64b14b3b10ebf8d8a75fe => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:32.007863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6e230-b002-4f49-a3ba-8bf8e6a2eccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:32.008493 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6e230-b002-4f49-a3ba-8bf8e6a2eccd None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:32.008765 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6e230-b002-4f49-a3ba-8bf8e6a2eccd None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:32.009029 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6e230-b002-4f49-a3ba-8bf8e6a2eccd None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:32.009196 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46aafef-59cb-4d57-b43f-3fdd488be3f5 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:32.009691 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6e230-b002-4f49-a3ba-8bf8e6a2eccd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:32.009986 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6e230-b002-4f49-a3ba-8bf8e6a2eccd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:32.011403 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6e230-b002-4f49-a3ba-8bf8e6a2eccd None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:32.023865 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 158/331] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:32.030833 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6e230-b002-4f49-a3ba-8bf8e6a2eccd None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:32.047526 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:32.123594 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:32.130275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46aafef-59cb-4d57-b43f-3fdd488be3f5 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:32.160652 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6e230-b002-4f49-a3ba-8bf8e6a2eccd None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:32.172251 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840f596-6b54-4c8f-9490-87b9af601677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:32.172870 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840f596-6b54-4c8f-9490-87b9af601677 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:32.173142 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840f596-6b54-4c8f-9490-87b9af601677 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:32.173429 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840f596-6b54-4c8f-9490-87b9af601677 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:32.174144 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8840f596-6b54-4c8f-9490-87b9af601677 None admin] RBAC: Adding query filter params (group.id=593c2c6ed3e64b14b3b10ebf8d8a75fe, role.id=39eab44f330440ebbefe39e538462165, scope.project.id=ef9f41b2925a4825a22ae82a7bd2e955) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:32.174449 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8840f596-6b54-4c8f-9490-87b9af601677 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:32.175966 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8840f596-6b54-4c8f-9490-87b9af601677 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:32.184277 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 174/332] 10.222.0.30 () {56 vars in 1376 bytes} [Wed Nov 2 07:15:32 2022] GET /identity/v3/role_assignments?group.id=593c2c6ed3e64b14b3b10ebf8d8a75fe&scope.project.id=ef9f41b2925a4825a22ae82a7bd2e955&role.id=39eab44f330440ebbefe39e538462165 => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:32.204340 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c82373-3af1-418b-9899-d03839ff8056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:32.204881 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c82373-3af1-418b-9899-d03839ff8056 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:32.205146 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c82373-3af1-418b-9899-d03839ff8056 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:32.205412 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c82373-3af1-418b-9899-d03839ff8056 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:32.288623 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c82373-3af1-418b-9899-d03839ff8056 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:32.290129 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c82373-3af1-418b-9899-d03839ff8056 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:32.305805 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:32.407992 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 159/333] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:15:32.415147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840f596-6b54-4c8f-9490-87b9af601677 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:32.548906 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840f596-6b54-4c8f-9490-87b9af601677 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:32.584824 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647ceb1-413b-425b-a55e-12f3e8525a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:32.585461 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647ceb1-413b-425b-a55e-12f3e8525a01 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:32.585733 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647ceb1-413b-425b-a55e-12f3e8525a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:32.585999 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647ceb1-413b-425b-a55e-12f3e8525a01 None admin] PATH_INFO: `/v3/projects/b3f647c33aef4316a529d7730281cb6b` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:32.587158 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8647ceb1-413b-425b-a55e-12f3e8525a01 None admin] RBAC: Authorizing `identity:get_project(project_id=b3f647c33aef4316a529d7730281cb6b)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:32.588607 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8647ceb1-413b-425b-a55e-12f3e8525a01 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:32.590293 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 175/334] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 07:15:32 2022] GET /identity/v3/projects/b3f647c33aef4316a529d7730281cb6b => generated 344 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:32.596439 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c82373-3af1-418b-9899-d03839ff8056 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:32.711063 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c82373-3af1-418b-9899-d03839ff8056 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:32.727536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:32.761650 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d9650-3e7e-4291-841d-59da719560f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:32.762943 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d9650-3e7e-4291-841d-59da719560f6 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:32.763217 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d9650-3e7e-4291-841d-59da719560f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:32.763508 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d9650-3e7e-4291-841d-59da719560f6 None admin] PATH_INFO: `/v3/groups/8ed292cec73f4645a120339bd68f80ec` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:32.764979 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d9650-3e7e-4291-841d-59da719560f6 None admin] RBAC: Authorizing `identity:get_group(group_id=8ed292cec73f4645a120339bd68f80ec)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:32.766927 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d9650-3e7e-4291-841d-59da719560f6 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:32.768511 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 160/335] 10.222.0.30 () {56 vars in 1180 bytes} [Wed Nov 2 07:15:32 2022] GET /identity/v3/groups/8ed292cec73f4645a120339bd68f80ec => generated 229 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:32.776413 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647ceb1-413b-425b-a55e-12f3e8525a01 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:32.810978 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:32.889242 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647ceb1-413b-425b-a55e-12f3e8525a01 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:32.926168 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeea7d4-b8ca-471d-8b1f-582c8df92192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:32.926799 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeea7d4-b8ca-471d-8b1f-582c8df92192 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:32.927072 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeea7d4-b8ca-471d-8b1f-582c8df92192 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:32.927338 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeea7d4-b8ca-471d-8b1f-582c8df92192 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:32.928068 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeea7d4-b8ca-471d-8b1f-582c8df92192 None admin] RBAC: Adding query filter params (group.id=8ed292cec73f4645a120339bd68f80ec, role.id=1e35bb725eb8475ea63e2d9538ab67d5, scope.project.id=b3f647c33aef4316a529d7730281cb6b) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:32.928370 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeea7d4-b8ca-471d-8b1f-582c8df92192 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:32.929805 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeea7d4-b8ca-471d-8b1f-582c8df92192 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:32.935695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 176/336] 10.222.0.30 () {56 vars in 1376 bytes} [Wed Nov 2 07:15:32 2022] GET /identity/v3/role_assignments?group.id=8ed292cec73f4645a120339bd68f80ec&scope.project.id=b3f647c33aef4316a529d7730281cb6b&role.id=1e35bb725eb8475ea63e2d9538ab67d5 => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:35.544968 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0ec8d-d547-43cc-8c9b-0fd9ae1cf488 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:35.544968 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0ec8d-d547-43cc-8c9b-0fd9ae1cf488 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:35.544968 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0ec8d-d547-43cc-8c9b-0fd9ae1cf488 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:35.546243 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 161/337] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:35.556473 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc9939-26bd-4d9c-b587-a6edbb29971a None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:35.556961 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc9939-26bd-4d9c-b587-a6edbb29971a None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:35.557041 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc9939-26bd-4d9c-b587-a6edbb29971a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:35.584605 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-c8cc9939-26bd-4d9c-b587-a6edbb29971a None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:35.585901 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-c8cc9939-26bd-4d9c-b587-a6edbb29971a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:35.672986 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 177/338] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:15:35.707039 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e919670-fa3b-4096-8636-a6d4a9f1d9b6 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:35.707685 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e919670-fa3b-4096-8636-a6d4a9f1d9b6 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:35.708168 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e919670-fa3b-4096-8636-a6d4a9f1d9b6 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:35.709937 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 162/339] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:35.715537 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344c776-e37a-4313-a362-8647e60f4c8c None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:35.716194 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344c776-e37a-4313-a362-8647e60f4c8c None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:35.716667 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344c776-e37a-4313-a362-8647e60f4c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:35.748224 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-b344c776-e37a-4313-a362-8647e60f4c8c None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:35.749538 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-b344c776-e37a-4313-a362-8647e60f4c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:35.769584 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:35.837510 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 178/340] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:35.986819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d3fbe-4fe2-4890-a35c-4bd8957e260c None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:35.987510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d3fbe-4fe2-4890-a35c-4bd8957e260c None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:35.987997 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d3fbe-4fe2-4890-a35c-4bd8957e260c None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:35.989894 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 163/341] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:35.995427 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55d463-2e09-490b-8c17-ef0a879b5495 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:35.995966 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55d463-2e09-490b-8c17-ef0a879b5495 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:35.996441 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55d463-2e09-490b-8c17-ef0a879b5495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:36.004003 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d3fbe-4fe2-4890-a35c-4bd8957e260c None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:36.026969 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-ac55d463-2e09-490b-8c17-ef0a879b5495 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:36.028555 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-ac55d463-2e09-490b-8c17-ef0a879b5495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:36.146396 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 179/342] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:36.164942 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d3fbe-4fe2-4890-a35c-4bd8957e260c None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:36.178387 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac55d463-2e09-490b-8c17-ef0a879b5495 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:36.230380 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8880c6bb-b886-483c-8ac8-b054988a87a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:36.231134 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880c6bb-b886-483c-8ac8-b054988a87a6 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:36.231424 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880c6bb-b886-483c-8ac8-b054988a87a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:36.231685 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880c6bb-b886-483c-8ac8-b054988a87a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:36.251241 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8880c6bb-b886-483c-8ac8-b054988a87a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:36.252704 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8880c6bb-b886-483c-8ac8-b054988a87a6 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:36.282543 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-8880c6bb-b886-483c-8ac8-b054988a87a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:36.282543 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:36.282543 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:36.282543 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:36.282543 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:36.282543 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:36.282543 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:36.282543 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:36.282543 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:36.282543 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:36.282543 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:36.282543 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:36.282543 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:36.282543 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:36.282543 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:36.282543 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:36.283581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:36.283581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:36.283581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:36.283581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:36.283581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:36.283581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:36.283581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:36.283581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:36.283581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:36.283581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:36.283581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:36.283581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:36.283581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:36.283581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:36.283581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:36.283581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:36.283581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:36.284449 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:36.284449 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:36.284449 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:36.284449 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 164/343] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:36.291629 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8880c6bb-b886-483c-8ac8-b054988a87a6 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:36.321858 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:36.346463 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:36.350000 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac55d463-2e09-490b-8c17-ef0a879b5495 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:36.406954 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673aebc1-2b11-43dd-b772-c6263a464901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:36.407512 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673aebc1-2b11-43dd-b772-c6263a464901 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:36.407959 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673aebc1-2b11-43dd-b772-c6263a464901 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:36.408391 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673aebc1-2b11-43dd-b772-c6263a464901 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:36.419176 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673aebc1-2b11-43dd-b772-c6263a464901 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:36.421136 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673aebc1-2b11-43dd-b772-c6263a464901 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:36.444660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-673aebc1-2b11-43dd-b772-c6263a464901 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:36.444660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:36.444660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:36.444660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:36.444660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:36.444660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:36.444660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:36.444660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:36.444660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:36.444660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:36.444660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:36.444660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:36.444660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:36.444660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:36.444660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:36.444660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:36.445811 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:36.445811 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:36.445811 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:36.445811 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:36.445811 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:36.445811 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:36.445811 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:36.445811 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:36.445811 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:36.445811 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:36.445811 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:36.445811 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:36.445811 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:36.445811 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:36.445811 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:36.445811 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:36.445811 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:36.446954 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:36.446954 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:36.446954 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:36.448916 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 180/344] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:36.450167 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673aebc1-2b11-43dd-b772-c6263a464901 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:36.473841 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8880c6bb-b886-483c-8ac8-b054988a87a6 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:36.514442 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb9d54-b972-4226-962d-e61fa82f5f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:36.515197 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb9d54-b972-4226-962d-e61fa82f5f69 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:36.515648 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb9d54-b972-4226-962d-e61fa82f5f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:36.515918 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb9d54-b972-4226-962d-e61fa82f5f69 None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:36.516977 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb9d54-b972-4226-962d-e61fa82f5f69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:36.517290 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb9d54-b972-4226-962d-e61fa82f5f69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:36.518939 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb9d54-b972-4226-962d-e61fa82f5f69 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:36.538974 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 165/345] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:36.539918 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb9d54-b972-4226-962d-e61fa82f5f69 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:36.566589 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673aebc1-2b11-43dd-b772-c6263a464901 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:36.610221 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff92071-94a3-4b3a-971d-e81bf100b300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:36.610893 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff92071-94a3-4b3a-971d-e81bf100b300 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:36.611172 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff92071-94a3-4b3a-971d-e81bf100b300 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:36.611441 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff92071-94a3-4b3a-971d-e81bf100b300 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:36.614344 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:36.624083 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff92071-94a3-4b3a-971d-e81bf100b300 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:36.625563 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff92071-94a3-4b3a-971d-e81bf100b300 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:36.641170 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-3ff92071-94a3-4b3a-971d-e81bf100b300 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:36.641170 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:36.641170 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:36.641170 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:36.641170 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:36.641170 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:36.641170 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:36.641170 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:36.641170 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:36.641170 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:36.641170 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:36.641170 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:36.641170 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:36.641170 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:36.641170 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:36.641170 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:36.642141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:36.642141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:36.642141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:36.642141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:36.642141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:36.642141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:36.642141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:36.642141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:36.642141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:36.642141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:36.642141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:36.642141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:36.642141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:36.642141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:36.642141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:36.642141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:36.642141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:36.642990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:36.642990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:36.642990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:36.642990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 181/346] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:36.643856 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff92071-94a3-4b3a-971d-e81bf100b300 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:36.645136 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:36.657135 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb9d54-b972-4226-962d-e61fa82f5f69 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:36.696393 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd12f13d-483a-436b-bbbb-426d3a1b093f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:36.697008 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12f13d-483a-436b-bbbb-426d3a1b093f None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:36.697387 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12f13d-483a-436b-bbbb-426d3a1b093f None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:36.697548 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12f13d-483a-436b-bbbb-426d3a1b093f None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:36.698189 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd12f13d-483a-436b-bbbb-426d3a1b093f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:36.698488 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd12f13d-483a-436b-bbbb-426d3a1b093f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:36.699902 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd12f13d-483a-436b-bbbb-426d3a1b093f None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:36.712125 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 166/347] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:36.713429 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3fbe1c-9c79-40dd-b50a-22f0f533fab5 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:36.713723 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3fbe1c-9c79-40dd-b50a-22f0f533fab5 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:36.713987 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3fbe1c-9c79-40dd-b50a-22f0f533fab5 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:36.718786 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 167/348] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:36.718786 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3fbe1c-9c79-40dd-b50a-22f0f533fab5 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:36.756992 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff92071-94a3-4b3a-971d-e81bf100b300 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:36.792556 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae5238-c330-4f07-a77e-aab3cc0fa778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:36.793214 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae5238-c330-4f07-a77e-aab3cc0fa778 None admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:36.793488 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae5238-c330-4f07-a77e-aab3cc0fa778 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:36.793851 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae5238-c330-4f07-a77e-aab3cc0fa778 None admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:36.794490 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae5238-c330-4f07-a77e-aab3cc0fa778 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:36.796025 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae5238-c330-4f07-a77e-aab3cc0fa778 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:36.817642 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 182/349] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 07:15:36 2022] POST /identity/v3/users => generated 246 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:15:36.828106 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae5238-c330-4f07-a77e-aab3cc0fa778 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:36.834594 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3fbe1c-9c79-40dd-b50a-22f0f533fab5 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:36.874095 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b6131-f18f-49b1-a66d-4e8a055b8c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:36.874877 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b6131-f18f-49b1-a66d-4e8a055b8c75 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:36.875273 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b6131-f18f-49b1-a66d-4e8a055b8c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:36.875609 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b6131-f18f-49b1-a66d-4e8a055b8c75 None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:36.876389 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357b6131-f18f-49b1-a66d-4e8a055b8c75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:36.876761 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357b6131-f18f-49b1-a66d-4e8a055b8c75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:36.878367 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357b6131-f18f-49b1-a66d-4e8a055b8c75 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:36.897166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 168/350] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:36.900263 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6838c3-1b99-43ce-9404-44503840e3a9 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:36.900833 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6838c3-1b99-43ce-9404-44503840e3a9 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:36.901390 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6838c3-1b99-43ce-9404-44503840e3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:36.905185 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:36.930899 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-df6838c3-1b99-43ce-9404-44503840e3a9 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:36.932079 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-df6838c3-1b99-43ce-9404-44503840e3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:36.941166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:36.994423 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae5238-c330-4f07-a77e-aab3cc0fa778 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:37.018508 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 169/351] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:37.023504 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6838c3-1b99-43ce-9404-44503840e3a9 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:37.042127 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f2efd-66c8-462a-8104-76973f3b6202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:37.042789 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f2efd-66c8-462a-8104-76973f3b6202 None admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:37.043116 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f2efd-66c8-462a-8104-76973f3b6202 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:37.043390 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f2efd-66c8-462a-8104-76973f3b6202 None admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:37.044416 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44f2efd-66c8-462a-8104-76973f3b6202 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:37.046070 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44f2efd-66c8-462a-8104-76973f3b6202 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:37.071579 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 183/352] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 07:15:36 2022] POST /identity/v3/users => generated 248 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:15:37.130424 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eb0574-dace-4866-95a4-da7d4a88399c None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:37.130812 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eb0574-dace-4866-95a4-da7d4a88399c None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:37.131016 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eb0574-dace-4866-95a4-da7d4a88399c None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:37.132718 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 184/353] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:15:37.139086 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77212986-76cd-435d-a470-7234d79d1b44 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:37.139427 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77212986-76cd-435d-a470-7234d79d1b44 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:37.139644 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77212986-76cd-435d-a470-7234d79d1b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:37.154388 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6838c3-1b99-43ce-9404-44503840e3a9 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:37.160769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-77212986-76cd-435d-a470-7234d79d1b44 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:37.161990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-77212986-76cd-435d-a470-7234d79d1b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:37.200550 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b80b113-5a46-47b7-9a46-2e5e8848a3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:37.201860 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80b113-5a46-47b7-9a46-2e5e8848a3e4 None admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:37.202799 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80b113-5a46-47b7-9a46-2e5e8848a3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:37.203434 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80b113-5a46-47b7-9a46-2e5e8848a3e4 None admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:37.204548 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b80b113-5a46-47b7-9a46-2e5e8848a3e4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:37.206465 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b80b113-5a46-47b7-9a46-2e5e8848a3e4 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:37.253586 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 170/354] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 07:15:37 2022] POST /identity/v3/users => generated 248 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:15:37.254262 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:37.261432 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 185/355] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:15:37.331636 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b80b113-5a46-47b7-9a46-2e5e8848a3e4 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:37.473192 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b80b113-5a46-47b7-9a46-2e5e8848a3e4 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:37.509158 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9f43c-042f-40a9-ad2a-43fead589cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:37.509841 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9f43c-042f-40a9-ad2a-43fead589cd3 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:37.510111 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9f43c-042f-40a9-ad2a-43fead589cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:37.510420 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9f43c-042f-40a9-ad2a-43fead589cd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:37.522904 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9f43c-042f-40a9-ad2a-43fead589cd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:37.524345 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9f43c-042f-40a9-ad2a-43fead589cd3 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:37.539781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-a5a9f43c-042f-40a9-ad2a-43fead589cd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:37.539781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:37.539781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:37.539781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:37.539781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:37.539781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:37.539781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:37.539781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:37.539781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:37.539781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:37.539781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:37.539781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:37.539781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:37.539781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:37.539781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:37.539781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:37.540729 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:37.540729 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:37.540729 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:37.540729 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:37.540729 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:37.540729 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:37.540729 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:37.540729 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:37.540729 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:37.540729 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:37.540729 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:37.540729 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:37.540729 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:37.540729 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:37.540729 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:37.540729 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:37.540729 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:37.541554 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:37.541554 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:37.541554 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:37.541554 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 171/356] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:37.548234 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77212986-76cd-435d-a470-7234d79d1b44 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:37.566836 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9f43c-042f-40a9-ad2a-43fead589cd3 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:37.586471 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:37.647050 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:37.689692 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9f43c-042f-40a9-ad2a-43fead589cd3 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:37.716889 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77212986-76cd-435d-a470-7234d79d1b44 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:37.730768 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5220b7-c239-4f50-9148-19379ca8584e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:37.730768 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5220b7-c239-4f50-9148-19379ca8584e None admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:37.730768 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5220b7-c239-4f50-9148-19379ca8584e None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:37.730768 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5220b7-c239-4f50-9148-19379ca8584e None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:37.731196 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5220b7-c239-4f50-9148-19379ca8584e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:37.732257 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5220b7-c239-4f50-9148-19379ca8584e None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:37.741630 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.sql.core [None req-eb5220b7-c239-4f50-9148-19379ca8584e None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Nov 02 07:15:37.741630 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Nov 02 07:15:37.741630 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [parameters: {'id': '7c2589f471c14012ac8c0305f87b9566', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Nov 02 07:15:37.741630 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81548) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Nov 02 07:15:37.751278 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-eb5220b7-c239-4f50-9148-19379ca8584e None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Nov 02 07:15:37.751278 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:37.751278 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:15:37.751278 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:15:37.751278 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:15:37.751278 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:15:37.751278 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:15:37.751278 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:15:37.751278 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:15:37.751278 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:15:37.751278 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:15:37.751278 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:15:37.751278 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:15:37.751278 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application result.read() Nov 02 07:15:37.751278 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:15:37.752193 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:15:37.752193 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:15:37.752193 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:15:37.752193 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:15:37.752193 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:15:37.752193 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:15:37.752193 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:15:37.752193 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Nov 02 07:15:37.752193 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:37.752193 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Nov 02 07:15:37.752193 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:37.752193 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:37.752193 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Nov 02 07:15:37.752193 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return method(*args, **kwargs) Nov 02 07:15:37.752193 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Nov 02 07:15:37.752193 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return ref.to_dict() Nov 02 07:15:37.752193 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:15:37.752193 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:15:37.753431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Nov 02 07:15:37.753431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application yield resource Nov 02 07:15:37.753431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:15:37.753431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:15:37.753431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Nov 02 07:15:37.753431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self.session.rollback() Nov 02 07:15:37.753431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Nov 02 07:15:37.753431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self.force_reraise() Nov 02 07:15:37.753431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Nov 02 07:15:37.753431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise self.value Nov 02 07:15:37.753431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Nov 02 07:15:37.753431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Nov 02 07:15:37.753431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Nov 02 07:15:37.753431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application session.commit() Nov 02 07:15:37.753431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Nov 02 07:15:37.753431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Nov 02 07:15:37.753431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Nov 02 07:15:37.754470 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._prepare_impl() Nov 02 07:15:37.754470 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Nov 02 07:15:37.754470 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self.session.flush() Nov 02 07:15:37.754470 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Nov 02 07:15:37.754470 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._flush(objects) Nov 02 07:15:37.754470 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Nov 02 07:15:37.754470 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Nov 02 07:15:37.754470 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Nov 02 07:15:37.754470 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application compat.raise_( Nov 02 07:15:37.754470 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:15:37.754470 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception Nov 02 07:15:37.754470 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Nov 02 07:15:37.754470 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application flush_context.execute() Nov 02 07:15:37.754470 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Nov 02 07:15:37.754470 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rec.execute(self) Nov 02 07:15:37.754470 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Nov 02 07:15:37.754470 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Nov 02 07:15:37.755340 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Nov 02 07:15:37.755340 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application _emit_insert_statements( Nov 02 07:15:37.755340 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Nov 02 07:15:37.755340 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application c = connection._execute_20( Nov 02 07:15:37.755340 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Nov 02 07:15:37.755340 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Nov 02 07:15:37.755340 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Nov 02 07:15:37.755340 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Nov 02 07:15:37.755340 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Nov 02 07:15:37.755340 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ret = self._execute_context( Nov 02 07:15:37.755340 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Nov 02 07:15:37.755340 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Nov 02 07:15:37.755340 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Nov 02 07:15:37.755340 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Nov 02 07:15:37.755340 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:15:37.755340 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception Nov 02 07:15:37.756139 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:15:37.756139 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:15:37.756139 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:15:37.756139 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:15:37.756139 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:15:37.756139 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:15:37.756139 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:15:37.756139 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:15:37.756139 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:15:37.756139 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:15:37.756139 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:15:37.756139 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application result.read() Nov 02 07:15:37.756139 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:15:37.756139 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:15:37.756139 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:15:37.756139 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:15:37.756899 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:15:37.756899 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:15:37.756899 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:15:37.756899 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:15:37.756899 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Nov 02 07:15:37.756899 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Nov 02 07:15:37.756899 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [parameters: {'id': '7c2589f471c14012ac8c0305f87b9566', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Nov 02 07:15:37.756899 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Nov 02 07:15:37.756899 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:37.756899 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:15:37.756899 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:37.756899 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:37.756899 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:37.756899 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:37.756899 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:37.756899 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:37.758882 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:37.758882 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:37.758882 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:37.758882 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:37.758882 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:37.758882 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:37.758882 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Nov 02 07:15:37.758882 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Nov 02 07:15:37.758882 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:37.758882 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:37.758882 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Nov 02 07:15:37.758882 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Nov 02 07:15:37.758882 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Nov 02 07:15:37.758882 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Nov 02 07:15:37.758882 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Nov 02 07:15:37.758882 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:37.760664 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 172/357] 10.222.0.30 () {60 vars in 1131 bytes} [Wed Nov 2 07:15:37 2022] POST /identity/v3/roles => generated 144 bytes in 187 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:37.760664 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5220b7-c239-4f50-9148-19379ca8584e None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:37.805190 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674f76e1-5896-4f26-86a1-e5696fe58f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:37.805841 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f76e1-5896-4f26-86a1-e5696fe58f14 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:37.806211 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f76e1-5896-4f26-86a1-e5696fe58f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:37.806488 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f76e1-5896-4f26-86a1-e5696fe58f14 None admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:37.807206 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674f76e1-5896-4f26-86a1-e5696fe58f14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:37.807507 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674f76e1-5896-4f26-86a1-e5696fe58f14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:37.809035 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674f76e1-5896-4f26-86a1-e5696fe58f14 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:37.856183 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 186/358] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:37.859739 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674f76e1-5896-4f26-86a1-e5696fe58f14 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:37.918351 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5220b7-c239-4f50-9148-19379ca8584e None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:37.950756 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:37.996503 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b609b-2786-4081-bf29-8a68438fba8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:37.998988 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b609b-2786-4081-bf29-8a68438fba8f None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:37.999264 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b609b-2786-4081-bf29-8a68438fba8f None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:37.999529 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b609b-2786-4081-bf29-8a68438fba8f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:38.021922 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b609b-2786-4081-bf29-8a68438fba8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:38.023333 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-0c0b609b-2786-4081-bf29-8a68438fba8f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:15:38.023333 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:38.023333 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:38.023333 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:38.023333 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:38.023333 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:38.023333 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:38.023333 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:38.023333 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:38.023333 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:38.023333 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:38.023333 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:38.023333 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:38.023333 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:38.023333 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:38.023333 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:38.024212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:38.024212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:38.024212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:38.024212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:38.024212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:38.024212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:38.024212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:38.024212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:38.024212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:38.024212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:38.024212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:38.024212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:38.024212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:38.024212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:38.024212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:38.024212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:38.025718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:38.025718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:38.025718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:38.025718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:38.025718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:38.025718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:38.025718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:38.025718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:38.025718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:38.025718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:38.025718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 07:15:38.025718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:38.025718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 173/359] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Nov 2 07:15:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:38.031115 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b609b-2786-4081-bf29-8a68438fba8f None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:38.045721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:38.057851 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674f76e1-5896-4f26-86a1-e5696fe58f14 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:38.106243 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb36f69-6994-438c-8b79-7071a9e30fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:38.107401 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb36f69-6994-438c-8b79-7071a9e30fb3 None admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:38.107918 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb36f69-6994-438c-8b79-7071a9e30fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:38.108412 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb36f69-6994-438c-8b79-7071a9e30fb3 None admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:38.109351 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb36f69-6994-438c-8b79-7071a9e30fb3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:38.111040 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb36f69-6994-438c-8b79-7071a9e30fb3 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:38.137418 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b609b-2786-4081-bf29-8a68438fba8f None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:38.153100 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 187/360] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 07:15:37 2022] POST /identity/v3/users => generated 249 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:15:38.176920 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64991a35-f7ad-47a8-8a54-56928e1d11d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:38.177527 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64991a35-f7ad-47a8-8a54-56928e1d11d8 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:38.177800 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64991a35-f7ad-47a8-8a54-56928e1d11d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:38.178112 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64991a35-f7ad-47a8-8a54-56928e1d11d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:38.178810 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64991a35-f7ad-47a8-8a54-56928e1d11d8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:38.179110 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64991a35-f7ad-47a8-8a54-56928e1d11d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:38.180600 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64991a35-f7ad-47a8-8a54-56928e1d11d8 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:38.191330 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 174/361] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:39.533079 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3528e-7971-4ff9-87f8-3efc6b9dc75b None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:39.533615 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3528e-7971-4ff9-87f8-3efc6b9dc75b None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:39.533615 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3528e-7971-4ff9-87f8-3efc6b9dc75b None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:39.535254 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 188/362] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:39.540389 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a739b-ab3b-4fb4-9365-9031ff8775c6 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:39.540822 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a739b-ab3b-4fb4-9365-9031ff8775c6 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:39.540904 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a739b-ab3b-4fb4-9365-9031ff8775c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:39.559576 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-7b3a739b-ab3b-4fb4-9365-9031ff8775c6 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:39.561127 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-7b3a739b-ab3b-4fb4-9365-9031ff8775c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:39.593060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:39.636585 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 175/363] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:39.733479 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a6839-12d8-4fb1-9c5a-128347461156 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:39.733789 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a6839-12d8-4fb1-9c5a-128347461156 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:39.734011 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a6839-12d8-4fb1-9c5a-128347461156 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:39.735411 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 189/364] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:39.740702 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd1405-2eea-4df7-b6c2-418ff96347b4 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:39.740985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd1405-2eea-4df7-b6c2-418ff96347b4 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:39.741259 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd1405-2eea-4df7-b6c2-418ff96347b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:39.767708 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-3bfd1405-2eea-4df7-b6c2-418ff96347b4 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:39.768896 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-3bfd1405-2eea-4df7-b6c2-418ff96347b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:39.837680 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 176/365] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:40.050986 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450a6839-12d8-4fb1-9c5a-128347461156 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:40.168457 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfd1405-2eea-4df7-b6c2-418ff96347b4 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:40.174192 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450a6839-12d8-4fb1-9c5a-128347461156 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:40.219402 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2bfa26-8a39-48b8-aa6a-2a55331012be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:40.219994 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2bfa26-8a39-48b8-aa6a-2a55331012be None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:40.220277 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2bfa26-8a39-48b8-aa6a-2a55331012be None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:40.220532 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2bfa26-8a39-48b8-aa6a-2a55331012be None admin] PATH_INFO: `/v3/roles/service` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:40.234252 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2bfa26-8a39-48b8-aa6a-2a55331012be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:40.240538 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-0c2bfa26-8a39-48b8-aa6a-2a55331012be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:40.240538 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:40.240538 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:40.240538 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:40.240538 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:40.240538 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:40.240538 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:40.240538 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:40.240538 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:40.240538 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:40.240538 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:40.240538 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:40.240538 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:40.240538 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:40.240538 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:40.240538 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:40.242603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:40.242603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:40.242603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:40.242603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:40.242603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:40.242603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:40.242603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:40.242603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:40.242603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:40.242603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:40.242603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:40.242603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:40.242603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:40.242603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:40.242603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:40.242603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:40.243455 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:40.243455 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:40.243455 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:40.243455 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:40.243455 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:40.243455 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:40.243455 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:40.243455 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:40.243455 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:40.243455 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:40.243455 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:40.243455 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:40.243455 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 190/366] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Nov 2 07:15:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:40.244765 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd3f76-843d-48d5-b99e-4622793622cf None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:40.244765 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd3f76-843d-48d5-b99e-4622793622cf None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:40.244765 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd3f76-843d-48d5-b99e-4622793622cf None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:40.246761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 191/367] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:40.247785 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd3f76-843d-48d5-b99e-4622793622cf None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:40.271064 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:40.295157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:40.302577 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfd1405-2eea-4df7-b6c2-418ff96347b4 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:40.342104 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f5606-51ec-41cd-af13-59ac720878f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:40.343165 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f5606-51ec-41cd-af13-59ac720878f3 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:40.343643 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f5606-51ec-41cd-af13-59ac720878f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:40.344115 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f5606-51ec-41cd-af13-59ac720878f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:40.355031 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77f5606-51ec-41cd-af13-59ac720878f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:40.360677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-a77f5606-51ec-41cd-af13-59ac720878f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:40.360677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:40.360677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:40.360677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:40.360677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:40.360677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:40.360677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:40.360677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:40.360677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:40.360677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:40.360677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:40.360677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:40.360677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:40.360677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:40.360677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:40.360677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:40.361553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:40.361553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:40.361553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:40.361553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:40.361553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:40.361553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:40.361553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:40.361553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:40.361553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:40.361553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:40.361553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:40.361553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:40.361553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:40.361553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:40.361553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:40.361553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:40.362338 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:40.362338 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:40.362338 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:40.362338 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:40.362338 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:40.362338 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:40.362338 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:40.362338 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:40.362338 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:40.362338 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:40.362338 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:40.362338 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:40.365015 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 177/368] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Nov 2 07:15:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:40.367002 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e33a61-e771-4eba-87aa-6767624ec423 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:40.367494 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e33a61-e771-4eba-87aa-6767624ec423 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:40.367954 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e33a61-e771-4eba-87aa-6767624ec423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:40.383092 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd3f76-843d-48d5-b99e-4622793622cf None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:40.387282 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-24e33a61-e771-4eba-87aa-6767624ec423 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:40.388725 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-24e33a61-e771-4eba-87aa-6767624ec423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:40.426499 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f765001-87d3-4155-bd6b-09feb6bd956b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:40.427055 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f765001-87d3-4155-bd6b-09feb6bd956b None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:40.427329 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f765001-87d3-4155-bd6b-09feb6bd956b None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:40.427594 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f765001-87d3-4155-bd6b-09feb6bd956b None admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:40.428528 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f765001-87d3-4155-bd6b-09feb6bd956b None admin] RBAC: Adding query filter params (name=service) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:40.428828 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f765001-87d3-4155-bd6b-09feb6bd956b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:40.431225 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f765001-87d3-4155-bd6b-09feb6bd956b None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:40.442026 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 192/369] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Nov 2 07:15:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:40.443370 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f765001-87d3-4155-bd6b-09feb6bd956b None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:40.474769 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 178/370] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:40.478463 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e33a61-e771-4eba-87aa-6767624ec423 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:40.557946 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:40.630968 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f765001-87d3-4155-bd6b-09feb6bd956b None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:40.652568 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:40.672230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e33a61-e771-4eba-87aa-6767624ec423 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:40.705064 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856ebe7-5503-4ead-959a-a9e7158859e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:40.707430 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856ebe7-5503-4ead-959a-a9e7158859e6 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:40.707710 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856ebe7-5503-4ead-959a-a9e7158859e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:40.707980 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856ebe7-5503-4ead-959a-a9e7158859e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:40.708683 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0856ebe7-5503-4ead-959a-a9e7158859e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:40.708980 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0856ebe7-5503-4ead-959a-a9e7158859e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:40.710414 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0856ebe7-5503-4ead-959a-a9e7158859e6 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:40.720889 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 193/371] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Nov 2 07:15:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:40.727632 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856ebe7-5503-4ead-959a-a9e7158859e6 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:40.758120 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0b7fb-02a3-4e9c-a5d8-d42c55c0e2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:40.759153 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0b7fb-02a3-4e9c-a5d8-d42c55c0e2b7 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:40.759634 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0b7fb-02a3-4e9c-a5d8-d42c55c0e2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:40.760098 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0b7fb-02a3-4e9c-a5d8-d42c55c0e2b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:40.783539 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0b7fb-02a3-4e9c-a5d8-d42c55c0e2b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:40.787135 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0b7fb-02a3-4e9c-a5d8-d42c55c0e2b7 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:40.809275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-fbd0b7fb-02a3-4e9c-a5d8-d42c55c0e2b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:40.809275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:40.809275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:40.809275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:40.809275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:40.809275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:40.809275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:40.809275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:40.809275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:40.809275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:40.809275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:40.809275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:40.809275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:40.809275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:40.809275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:40.809275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:40.810676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:40.810676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:40.810676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:40.810676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:40.810676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:40.810676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:40.810676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:40.810676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:40.810676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:40.810676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:40.810676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:40.810676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:40.810676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:40.810676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:40.810676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:40.810676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:40.810676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:40.811744 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:40.811744 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:40.811744 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:40.813158 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 179/372] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:40.813976 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0b7fb-02a3-4e9c-a5d8-d42c55c0e2b7 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:40.858546 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856ebe7-5503-4ead-959a-a9e7158859e6 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:40.892371 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:40.901118 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885708e-8f40-4079-88d6-06ba3740eb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:40.901754 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885708e-8f40-4079-88d6-06ba3740eb94 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:40.902030 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885708e-8f40-4079-88d6-06ba3740eb94 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:40.902297 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885708e-8f40-4079-88d6-06ba3740eb94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:40.914598 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8885708e-8f40-4079-88d6-06ba3740eb94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:40.916098 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8885708e-8f40-4079-88d6-06ba3740eb94 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:40.920789 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0b7fb-02a3-4e9c-a5d8-d42c55c0e2b7 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:40.932871 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-8885708e-8f40-4079-88d6-06ba3740eb94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:40.932871 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:40.932871 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:40.932871 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:40.932871 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:40.932871 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:40.932871 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:40.932871 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:40.932871 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:40.932871 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:40.932871 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:40.932871 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:40.932871 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:40.932871 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:40.932871 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:40.932871 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:40.934048 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:40.934048 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:40.934048 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:40.934048 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:40.934048 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:40.934048 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:40.934048 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:40.934048 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:40.934048 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:40.934048 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:40.934048 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:40.934048 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:40.934048 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:40.934048 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:40.934048 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:40.934048 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:40.934048 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:40.935234 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:40.935234 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:40.935234 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:40.935234 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 194/373] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:40.936291 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91c169-7e21-47e2-bbab-aa6a30301f75 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:40.936564 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91c169-7e21-47e2-bbab-aa6a30301f75 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:40.936826 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91c169-7e21-47e2-bbab-aa6a30301f75 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:40.938256 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 195/374] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:40.942908 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91c169-7e21-47e2-bbab-aa6a30301f75 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:40.962848 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9923b-f430-42a8-ad5f-0590d8977a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:40.963391 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9923b-f430-42a8-ad5f-0590d8977a3a None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:40.963662 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9923b-f430-42a8-ad5f-0590d8977a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:40.963927 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9923b-f430-42a8-ad5f-0590d8977a3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:40.971048 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:40.976049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9923b-f430-42a8-ad5f-0590d8977a3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:40.981207 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-31d9923b-f430-42a8-ad5f-0590d8977a3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:40.981207 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:40.981207 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:40.981207 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:40.981207 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:40.981207 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:40.981207 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:40.981207 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:40.981207 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:40.981207 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:40.981207 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:40.981207 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:40.981207 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:40.981207 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:40.981207 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:40.981207 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:40.982187 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:40.982187 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:40.982187 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:40.982187 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:40.982187 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:40.982187 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:40.982187 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:40.982187 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:40.982187 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:40.982187 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:40.982187 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:40.982187 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:40.982187 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:40.982187 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:40.982187 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:40.982187 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:40.983661 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:40.983661 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:40.983661 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:40.983661 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:40.983661 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:40.983661 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:40.983661 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:40.983661 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:40.983661 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:40.983661 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:40.983661 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:40.983661 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:40.983661 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 180/375] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Nov 2 07:15:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:40.983661 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9923b-f430-42a8-ad5f-0590d8977a3a None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:41.050832 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91c169-7e21-47e2-bbab-aa6a30301f75 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:41.083206 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920230e-366b-4ae9-a8d0-9717c1c06152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:41.083842 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920230e-366b-4ae9-a8d0-9717c1c06152 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:41.084121 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920230e-366b-4ae9-a8d0-9717c1c06152 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:41.084394 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920230e-366b-4ae9-a8d0-9717c1c06152 None admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:41.084872 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9923b-f430-42a8-ad5f-0590d8977a3a None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:41.085030 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6920230e-366b-4ae9-a8d0-9717c1c06152 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:41.085332 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6920230e-366b-4ae9-a8d0-9717c1c06152 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:41.086745 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6920230e-366b-4ae9-a8d0-9717c1c06152 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:41.097965 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 196/376] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:41.099234 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee0ac8-d1d0-4594-b8cc-4dbca8486783 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:41.099522 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee0ac8-d1d0-4594-b8cc-4dbca8486783 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:41.099797 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee0ac8-d1d0-4594-b8cc-4dbca8486783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:41.117819 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-22ee0ac8-d1d0-4594-b8cc-4dbca8486783 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:41.118347 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5625335c-58d4-4667-ba8c-9da49a7119cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:41.118984 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625335c-58d4-4667-ba8c-9da49a7119cf None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:41.119128 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-22ee0ac8-d1d0-4594-b8cc-4dbca8486783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:41.119215 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625335c-58d4-4667-ba8c-9da49a7119cf None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:41.119477 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625335c-58d4-4667-ba8c-9da49a7119cf None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:41.120137 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5625335c-58d4-4667-ba8c-9da49a7119cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:41.120433 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5625335c-58d4-4667-ba8c-9da49a7119cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:41.121776 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5625335c-58d4-4667-ba8c-9da49a7119cf None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:41.133232 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 181/377] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:41.138129 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5625335c-58d4-4667-ba8c-9da49a7119cf None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:41.154247 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:41.189139 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 197/378] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:41.189139 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4917b51-56a1-4bc7-b34f-bb2a19af304d None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:41.189139 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4917b51-56a1-4bc7-b34f-bb2a19af304d None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:41.189139 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4917b51-56a1-4bc7-b34f-bb2a19af304d None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:41.190691 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 198/379] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:41.190691 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4917b51-56a1-4bc7-b34f-bb2a19af304d None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:41.199902 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:41.242159 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5625335c-58d4-4667-ba8c-9da49a7119cf None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:41.275944 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef8e13-6162-4821-a3ca-18cac6b74d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:41.276557 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef8e13-6162-4821-a3ca-18cac6b74d55 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:41.276830 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef8e13-6162-4821-a3ca-18cac6b74d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:41.277102 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef8e13-6162-4821-a3ca-18cac6b74d55 None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:41.277864 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef8e13-6162-4821-a3ca-18cac6b74d55 None admin] RBAC: Adding query filter params (name=service) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:41.278167 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef8e13-6162-4821-a3ca-18cac6b74d55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:41.279737 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef8e13-6162-4821-a3ca-18cac6b74d55 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:41.289662 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 182/380] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Nov 2 07:15:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:41.294321 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef8e13-6162-4821-a3ca-18cac6b74d55 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:41.307830 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4917b51-56a1-4bc7-b34f-bb2a19af304d None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:41.343126 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493dd228-9e71-47f5-8cb5-0849e03aa00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:41.343918 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493dd228-9e71-47f5-8cb5-0849e03aa00f None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:41.344039 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493dd228-9e71-47f5-8cb5-0849e03aa00f None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:41.344275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493dd228-9e71-47f5-8cb5-0849e03aa00f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:41.351291 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493dd228-9e71-47f5-8cb5-0849e03aa00f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:41.352776 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493dd228-9e71-47f5-8cb5-0849e03aa00f None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:41.365910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-493dd228-9e71-47f5-8cb5-0849e03aa00f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:15:41.365910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:41.365910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:41.365910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:41.365910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:41.365910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:41.365910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:41.365910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:41.365910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:41.365910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:41.365910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:41.365910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:41.365910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:41.365910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:41.365910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:41.365910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:41.366834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:41.366834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:41.366834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:41.366834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:41.366834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:41.366834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:41.366834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:41.366834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:41.366834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:41.366834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:41.366834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:41.366834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:41.366834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:41.366834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:41.366834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:41.366834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:41.366834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:41.366834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:41.367763 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:41.367763 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:41.367763 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:41.367763 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:41.367763 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:41.367763 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:41.367763 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:41.367763 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:41.367763 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:15:41.367763 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:41.367763 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 199/381] 10.222.0.30 () {56 vars in 1093 bytes} [Wed Nov 2 07:15:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:41.368310 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220dfe38-5bd6-48d1-9f85-695dfb41f201 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:41.368374 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220dfe38-5bd6-48d1-9f85-695dfb41f201 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:41.374834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220dfe38-5bd6-48d1-9f85-695dfb41f201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:41.389162 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-220dfe38-5bd6-48d1-9f85-695dfb41f201 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:41.390247 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-220dfe38-5bd6-48d1-9f85-695dfb41f201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:41.402413 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef8e13-6162-4821-a3ca-18cac6b74d55 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:41.417400 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:41.424608 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:41.440342 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ec291c-1618-471e-932f-606c9e055c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:41.440995 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec291c-1618-471e-932f-606c9e055c6e None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:41.441301 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec291c-1618-471e-932f-606c9e055c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:41.441599 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec291c-1618-471e-932f-606c9e055c6e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:41.450109 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ec291c-1618-471e-932f-606c9e055c6e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:41.451724 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ec291c-1618-471e-932f-606c9e055c6e None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:41.464976 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-d7ec291c-1618-471e-932f-606c9e055c6e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Nov 02 07:15:41.464976 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:41.464976 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:41.464976 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:41.464976 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:41.464976 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:41.464976 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:41.464976 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:41.464976 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:41.464976 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:41.464976 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:41.464976 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:41.464976 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:41.464976 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:41.464976 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:41.464976 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:41.466085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:41.466085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:41.466085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:41.466085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:41.466085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:41.466085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:41.466085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:41.466085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:41.466085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:41.466085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:41.466085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:41.466085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:41.466085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:41.466085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:41.466085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:41.466085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:41.466085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:41.466085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:41.467375 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:41.467375 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:41.467375 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:41.467375 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:41.467375 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:41.467375 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:41.467375 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:41.467375 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:41.467375 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Nov 02 07:15:41.467375 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:41.467375 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 183/382] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:41.467375 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ec291c-1618-471e-932f-606c9e055c6e None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:41.469891 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 200/383] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:41.471983 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220dfe38-5bd6-48d1-9f85-695dfb41f201 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:41.567781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ec291c-1618-471e-932f-606c9e055c6e None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:41.576514 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220dfe38-5bd6-48d1-9f85-695dfb41f201 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:41.610171 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e675a-e10b-4be0-9c9c-ed1e21a76c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:41.610804 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e675a-e10b-4be0-9c9c-ed1e21a76c3e None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:41.611137 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e675a-e10b-4be0-9c9c-ed1e21a76c3e None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:41.611441 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e675a-e10b-4be0-9c9c-ed1e21a76c3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:41.612542 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27a4df-2392-4d33-a440-ce0c4c1afe80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:41.613249 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27a4df-2392-4d33-a440-ce0c4c1afe80 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:41.613429 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27a4df-2392-4d33-a440-ce0c4c1afe80 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:41.613700 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27a4df-2392-4d33-a440-ce0c4c1afe80 None admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:41.614427 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e27a4df-2392-4d33-a440-ce0c4c1afe80 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:41.614766 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e27a4df-2392-4d33-a440-ce0c4c1afe80 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:41.616184 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e27a4df-2392-4d33-a440-ce0c4c1afe80 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:41.623784 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e675a-e10b-4be0-9c9c-ed1e21a76c3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:41.625461 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e675a-e10b-4be0-9c9c-ed1e21a76c3e None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:41.641709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-576e675a-e10b-4be0-9c9c-ed1e21a76c3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:41.641709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:41.641709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:41.641709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:41.641709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:41.641709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:41.641709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:41.641709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:41.641709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:41.641709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:41.641709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:41.641709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:41.641709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:41.641709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:41.641709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:41.641709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:41.642710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:41.642710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:41.642710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:41.642710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:41.642710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:41.642710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:41.642710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:41.642710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:41.642710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:41.642710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:41.642710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:41.642710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:41.642710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:41.642710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:41.642710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:41.642710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:41.642710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:41.643574 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:41.643574 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:41.643574 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:41.643574 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 184/384] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:41.643814 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e675a-e10b-4be0-9c9c-ed1e21a76c3e None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:41.658616 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 201/385] 10.222.0.30 () {56 vars in 1133 bytes} [Wed Nov 2 07:15:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:41.659831 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27a4df-2392-4d33-a440-ce0c4c1afe80 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:41.682443 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:41.685788 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:41.758844 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e675a-e10b-4be0-9c9c-ed1e21a76c3e None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:41.767077 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27a4df-2392-4d33-a440-ce0c4c1afe80 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:41.793679 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57052673-c63f-4ba7-8660-782350430dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:41.794328 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57052673-c63f-4ba7-8660-782350430dcf None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:41.794770 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57052673-c63f-4ba7-8660-782350430dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:41.795052 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57052673-c63f-4ba7-8660-782350430dcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:41.800660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8764c8-da14-4eae-bdbd-43fc69f88378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:41.801272 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8764c8-da14-4eae-bdbd-43fc69f88378 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:41.801601 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8764c8-da14-4eae-bdbd-43fc69f88378 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:41.801870 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8764c8-da14-4eae-bdbd-43fc69f88378 None admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:41.802598 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8764c8-da14-4eae-bdbd-43fc69f88378 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:41.802979 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8764c8-da14-4eae-bdbd-43fc69f88378 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:41.804429 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8764c8-da14-4eae-bdbd-43fc69f88378 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:41.805809 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57052673-c63f-4ba7-8660-782350430dcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:41.807289 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57052673-c63f-4ba7-8660-782350430dcf None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:41.821607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-57052673-c63f-4ba7-8660-782350430dcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:41.821607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:41.821607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:41.821607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:41.821607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:41.821607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:41.821607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:41.821607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:41.821607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:41.821607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:41.821607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:41.821607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:41.821607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:41.821607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:41.821607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:41.821607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:41.822478 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:41.822478 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:41.822478 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:41.822478 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:41.822478 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:41.822478 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:41.822478 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:41.822478 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:41.822478 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:41.822478 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:41.822478 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:41.822478 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:41.822478 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:41.822478 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:41.822478 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:41.822478 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:41.822478 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:41.823357 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:41.823357 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:41.823357 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:41.823357 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 185/386] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:41.823578 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57052673-c63f-4ba7-8660-782350430dcf None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:41.823640 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 202/387] 10.222.0.30 () {56 vars in 1137 bytes} [Wed Nov 2 07:15:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:41.824354 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8764c8-da14-4eae-bdbd-43fc69f88378 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:41.902797 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:41.912591 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:41.925548 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8764c8-da14-4eae-bdbd-43fc69f88378 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:41.928539 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57052673-c63f-4ba7-8660-782350430dcf None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:41.959579 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41358a03-1462-49f9-8170-0962737ba23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:41.960209 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41358a03-1462-49f9-8170-0962737ba23c None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:41.960482 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41358a03-1462-49f9-8170-0962737ba23c None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:41.960755 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41358a03-1462-49f9-8170-0962737ba23c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:41.961790 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581feb8-2b2d-46f9-9dc0-be9da46f4a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:41.962375 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581feb8-2b2d-46f9-9dc0-be9da46f4a42 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:41.962670 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581feb8-2b2d-46f9-9dc0-be9da46f4a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:41.962938 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581feb8-2b2d-46f9-9dc0-be9da46f4a42 None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:41.963573 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7581feb8-2b2d-46f9-9dc0-be9da46f4a42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:41.963868 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7581feb8-2b2d-46f9-9dc0-be9da46f4a42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:41.965228 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7581feb8-2b2d-46f9-9dc0-be9da46f4a42 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:41.971420 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41358a03-1462-49f9-8170-0962737ba23c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:41.972919 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41358a03-1462-49f9-8170-0962737ba23c None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:41.976674 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 186/388] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:41.977477 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581feb8-2b2d-46f9-9dc0-be9da46f4a42 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:41.987547 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-41358a03-1462-49f9-8170-0962737ba23c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:41.987547 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:41.987547 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:41.987547 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:41.987547 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:41.987547 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:41.987547 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:41.987547 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:41.987547 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:41.987547 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:41.987547 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:41.987547 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:41.987547 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:41.987547 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:41.987547 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:41.987547 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:41.988654 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:41.988654 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:41.988654 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:41.988654 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:41.988654 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:41.988654 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:41.988654 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:41.988654 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:41.988654 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:41.988654 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:41.988654 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:41.988654 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:41.988654 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:41.988654 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:41.988654 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:41.988654 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:41.988654 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:41.989894 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:41.989894 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:41.989894 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:41.993145 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 203/389] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:41.993942 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41358a03-1462-49f9-8170-0962737ba23c None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:42.074046 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581feb8-2b2d-46f9-9dc0-be9da46f4a42 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:42.089219 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41358a03-1462-49f9-8170-0962737ba23c None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:42.107691 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fcf213-0ffe-4b07-9798-d97fab9fd23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:42.108311 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcf213-0ffe-4b07-9798-d97fab9fd23d None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:42.108579 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcf213-0ffe-4b07-9798-d97fab9fd23d None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:42.108844 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcf213-0ffe-4b07-9798-d97fab9fd23d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:42.117908 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:42.118862 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fcf213-0ffe-4b07-9798-d97fab9fd23d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:42.124138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-d2fcf213-0ffe-4b07-9798-d97fab9fd23d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:42.124138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:42.124138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:42.124138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:42.124138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:42.124138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:42.124138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:42.124138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:42.124138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:42.124138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:42.124138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:42.124138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:42.124138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:42.124138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:42.124138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:42.124138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:42.125036 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:42.125036 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:42.125036 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:42.125036 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:42.125036 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:42.125036 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:42.125036 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:42.125036 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:42.125036 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:42.125036 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:42.125036 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:42.125036 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:42.125036 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:42.125036 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:42.125036 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:42.125036 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:42.125819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:42.125819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:42.125819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:42.125819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:42.125819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:42.125819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:42.125819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:42.125819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:42.125819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:42.125819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:42.125819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:42.125819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:42.126408 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa504f-52c5-41a9-b88b-9b925d572e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:42.126408 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa504f-52c5-41a9-b88b-9b925d572e21 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:42.126408 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa504f-52c5-41a9-b88b-9b925d572e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:42.126408 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa504f-52c5-41a9-b88b-9b925d572e21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:42.126693 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 187/390] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Nov 2 07:15:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:42.126693 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fcf213-0ffe-4b07-9798-d97fab9fd23d None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:42.131589 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:42.139580 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfa504f-52c5-41a9-b88b-9b925d572e21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:42.141043 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfa504f-52c5-41a9-b88b-9b925d572e21 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:42.155034 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-ebfa504f-52c5-41a9-b88b-9b925d572e21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:42.155034 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:42.155034 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:42.155034 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:42.155034 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:42.155034 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:42.155034 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:42.155034 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:42.155034 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:42.155034 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:42.155034 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:42.155034 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:42.155034 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:42.155034 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:42.155034 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:42.155034 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:42.155990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:42.155990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:42.155990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:42.155990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:42.155990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:42.155990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:42.155990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:42.155990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:42.155990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:42.155990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:42.155990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:42.155990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:42.155990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:42.155990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:42.155990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:42.155990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:42.155990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:42.157301 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:42.157301 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:42.157301 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:42.157301 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 204/391] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:42.157301 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa504f-52c5-41a9-b88b-9b925d572e21 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:42.231316 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fcf213-0ffe-4b07-9798-d97fab9fd23d None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:42.255822 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa504f-52c5-41a9-b88b-9b925d572e21 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:42.265449 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c843b79-8975-4bb8-91b2-565771f4924f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:42.266055 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c843b79-8975-4bb8-91b2-565771f4924f None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:42.266283 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c843b79-8975-4bb8-91b2-565771f4924f None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:42.266559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c843b79-8975-4bb8-91b2-565771f4924f None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:42.267226 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c843b79-8975-4bb8-91b2-565771f4924f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:42.267524 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c843b79-8975-4bb8-91b2-565771f4924f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:42.268900 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c843b79-8975-4bb8-91b2-565771f4924f None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:42.280562 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 188/392] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:42.281618 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c843b79-8975-4bb8-91b2-565771f4924f None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:42.292300 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eee024-e80d-419a-b952-8d1611d7965b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:42.292989 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eee024-e80d-419a-b952-8d1611d7965b None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:42.293571 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eee024-e80d-419a-b952-8d1611d7965b None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:42.293842 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eee024-e80d-419a-b952-8d1611d7965b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:42.301658 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eee024-e80d-419a-b952-8d1611d7965b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:42.303162 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eee024-e80d-419a-b952-8d1611d7965b None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:42.314505 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-55eee024-e80d-419a-b952-8d1611d7965b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 07:15:42.314505 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:42.314505 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:42.314505 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:42.314505 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:42.314505 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:42.314505 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:42.314505 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:42.314505 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:42.314505 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:42.314505 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:42.314505 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:42.314505 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:42.314505 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:42.314505 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:42.314505 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:42.315571 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:42.315571 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:42.315571 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:42.315571 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:42.315571 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:42.315571 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:42.315571 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:42.315571 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:42.315571 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:42.315571 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:42.315571 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:42.315571 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:42.315571 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:42.315571 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:42.315571 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:42.315571 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:42.315571 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:42.315571 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:42.316679 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:42.316679 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:42.316679 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:42.316679 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:42.316679 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:42.316679 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:42.316679 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:42.316679 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:42.316679 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 07:15:42.316679 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:42.318442 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 205/393] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:42.319367 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eee024-e80d-419a-b952-8d1611d7965b None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:42.344346 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:42.363750 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:42.381023 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c843b79-8975-4bb8-91b2-565771f4924f None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:42.415552 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d196510-6fd3-4e1a-9209-6736fd67d730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:42.416143 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d196510-6fd3-4e1a-9209-6736fd67d730 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:42.416416 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d196510-6fd3-4e1a-9209-6736fd67d730 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:42.416684 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d196510-6fd3-4e1a-9209-6736fd67d730 None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:42.417325 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d196510-6fd3-4e1a-9209-6736fd67d730 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:42.417628 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d196510-6fd3-4e1a-9209-6736fd67d730 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:42.419050 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d196510-6fd3-4e1a-9209-6736fd67d730 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:42.422574 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eee024-e80d-419a-b952-8d1611d7965b None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:42.430130 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 189/394] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:42.430954 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d196510-6fd3-4e1a-9209-6736fd67d730 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:42.455895 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98672c-00bf-40b7-aa8e-8f9f6dd7c871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:42.456497 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98672c-00bf-40b7-aa8e-8f9f6dd7c871 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:42.456767 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98672c-00bf-40b7-aa8e-8f9f6dd7c871 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:42.457028 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98672c-00bf-40b7-aa8e-8f9f6dd7c871 None admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:42.457673 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba98672c-00bf-40b7-aa8e-8f9f6dd7c871 None admin] RBAC: Adding query filter params (name=service) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:42.457963 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba98672c-00bf-40b7-aa8e-8f9f6dd7c871 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:42.459350 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba98672c-00bf-40b7-aa8e-8f9f6dd7c871 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:42.467778 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 206/395] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Nov 2 07:15:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:42.467778 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98672c-00bf-40b7-aa8e-8f9f6dd7c871 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:42.532882 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d196510-6fd3-4e1a-9209-6736fd67d730 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:42.556497 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:42.568098 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6803c73-3fd1-4d55-aa99-f68ab393f47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:42.568708 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6803c73-3fd1-4d55-aa99-f68ab393f47e None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:42.568984 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6803c73-3fd1-4d55-aa99-f68ab393f47e None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:42.569255 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6803c73-3fd1-4d55-aa99-f68ab393f47e None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:42.569893 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6803c73-3fd1-4d55-aa99-f68ab393f47e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:42.570196 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6803c73-3fd1-4d55-aa99-f68ab393f47e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:42.571415 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98672c-00bf-40b7-aa8e-8f9f6dd7c871 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:42.571613 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6803c73-3fd1-4d55-aa99-f68ab393f47e None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:42.574057 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:42.583627 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 190/396] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:42.584391 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6803c73-3fd1-4d55-aa99-f68ab393f47e None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:42.611535 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e6e13-76e1-4b3f-8eba-528bb66520e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:42.612158 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e6e13-76e1-4b3f-8eba-528bb66520e7 None admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:42.612434 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e6e13-76e1-4b3f-8eba-528bb66520e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:42.612703 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e6e13-76e1-4b3f-8eba-528bb66520e7 None admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:42.613346 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145e6e13-76e1-4b3f-8eba-528bb66520e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:42.614841 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145e6e13-76e1-4b3f-8eba-528bb66520e7 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:42.636420 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 207/397] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 07:15:42 2022] POST /identity/v3/users => generated 247 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:15:42.637236 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e6e13-76e1-4b3f-8eba-528bb66520e7 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:42.696312 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6803c73-3fd1-4d55-aa99-f68ab393f47e None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:42.742790 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ce208-98b7-4183-983b-6028b45fc9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:42.743414 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ce208-98b7-4183-983b-6028b45fc9c4 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:42.743953 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ce208-98b7-4183-983b-6028b45fc9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:42.744233 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ce208-98b7-4183-983b-6028b45fc9c4 None admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:42.745328 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ce208-98b7-4183-983b-6028b45fc9c4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:42.745633 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ce208-98b7-4183-983b-6028b45fc9c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:42.747534 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ce208-98b7-4183-983b-6028b45fc9c4 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:42.761925 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e6e13-76e1-4b3f-8eba-528bb66520e7 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:42.792627 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 191/398] 10.222.0.30 () {56 vars in 1137 bytes} [Wed Nov 2 07:15:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:42.793583 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ce208-98b7-4183-983b-6028b45fc9c4 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:42.809539 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d195f59-0e79-4d2e-a997-243bff974478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:42.811276 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d195f59-0e79-4d2e-a997-243bff974478 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:42.811276 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d195f59-0e79-4d2e-a997-243bff974478 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:42.811276 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d195f59-0e79-4d2e-a997-243bff974478 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:42.818350 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d195f59-0e79-4d2e-a997-243bff974478 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:42.820202 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d195f59-0e79-4d2e-a997-243bff974478 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:42.834094 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-0d195f59-0e79-4d2e-a997-243bff974478 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:42.834094 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:42.834094 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:42.834094 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:42.834094 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:42.834094 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:42.834094 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:42.834094 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:42.834094 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:42.834094 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:42.834094 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:42.834094 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:42.834094 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:15:42.834094 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:15:42.834094 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:15:42.834973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:15:42.834973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:42.834973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:42.834973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:42.834973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:42.834973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:42.834973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:42.834973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:42.834973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:42.834973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:42.834973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:42.834973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:42.834973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:42.834973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:42.834973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:42.834973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:42.834973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:42.835812 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:15:42.835812 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:15:42.835812 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:15:42.835812 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:15:42.835812 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:15:42.835812 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:15:42.835812 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:42.835812 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:42.835812 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 208/399] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:42.836842 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d195f59-0e79-4d2e-a997-243bff974478 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:42.847379 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:42.862212 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:42.913208 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ce208-98b7-4183-983b-6028b45fc9c4 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:42.944331 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d195f59-0e79-4d2e-a997-243bff974478 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:42.950285 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2408d2-4c0a-4a6c-abc2-0742acb49bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:42.951021 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2408d2-4c0a-4a6c-abc2-0742acb49bed None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:42.951296 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2408d2-4c0a-4a6c-abc2-0742acb49bed None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:42.951962 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2408d2-4c0a-4a6c-abc2-0742acb49bed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:42.962967 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2408d2-4c0a-4a6c-abc2-0742acb49bed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:42.964591 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2408d2-4c0a-4a6c-abc2-0742acb49bed None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:42.978862 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff1966-7bcd-4f03-80a3-248ac7e81a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:42.979494 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff1966-7bcd-4f03-80a3-248ac7e81a34 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:42.979770 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff1966-7bcd-4f03-80a3-248ac7e81a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:42.980037 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff1966-7bcd-4f03-80a3-248ac7e81a34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:42.980236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-bd2408d2-4c0a-4a6c-abc2-0742acb49bed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:42.980236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:42.980236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:42.980236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:42.980236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:42.980236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:42.980236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:42.980236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:42.980236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:42.980236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:42.980236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:42.980236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:42.980236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:42.980236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:42.980236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:42.980236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:42.981077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:42.981077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:42.981077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:42.981077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:42.981077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:42.981077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:42.981077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:42.981077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:42.981077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:42.981077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:42.981077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:42.981077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:42.981077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:42.981077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:42.981077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:42.981077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:42.981077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:42.987193 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:42.987193 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:42.987193 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:42.987193 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 192/400] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:42.987193 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2408d2-4c0a-4a6c-abc2-0742acb49bed None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:42.992444 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff1966-7bcd-4f03-80a3-248ac7e81a34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:42.993959 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff1966-7bcd-4f03-80a3-248ac7e81a34 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:43.006154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-13ff1966-7bcd-4f03-80a3-248ac7e81a34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:43.006154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:43.006154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:43.006154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:43.006154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:43.006154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:43.006154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:43.006154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:43.006154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:43.006154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:43.006154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:43.006154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:43.006154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:15:43.006154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:15:43.006154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:15:43.007129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:15:43.007129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:43.007129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:43.007129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:43.007129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:43.007129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:43.007129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:43.007129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:43.007129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:43.007129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:43.007129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:43.007129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:43.007129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:43.007129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:43.007129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:43.007129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:43.007129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:43.007956 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:15:43.007956 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:15:43.007956 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:15:43.007956 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:15:43.007956 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:15:43.007956 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:15:43.007956 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:43.007956 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:43.007956 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 209/401] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:43.008415 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff1966-7bcd-4f03-80a3-248ac7e81a34 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:43.090238 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:43.093514 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:43.098342 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2408d2-4c0a-4a6c-abc2-0742acb49bed None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:43.116007 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff1966-7bcd-4f03-80a3-248ac7e81a34 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:43.134750 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aec5e0-d486-4226-9de0-048732814dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:43.135388 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aec5e0-d486-4226-9de0-048732814dec None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:43.135668 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aec5e0-d486-4226-9de0-048732814dec None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:43.135936 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aec5e0-d486-4226-9de0-048732814dec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:43.146849 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aec5e0-d486-4226-9de0-048732814dec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:43.148386 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aec5e0-d486-4226-9de0-048732814dec None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:43.153935 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a01cb-3aea-4783-9966-f4b229fb4101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:43.154930 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a01cb-3aea-4783-9966-f4b229fb4101 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:43.154930 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a01cb-3aea-4783-9966-f4b229fb4101 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:43.155212 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a01cb-3aea-4783-9966-f4b229fb4101 None admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:43.155836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83a01cb-3aea-4783-9966-f4b229fb4101 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:43.156475 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83a01cb-3aea-4783-9966-f4b229fb4101 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:43.157900 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83a01cb-3aea-4783-9966-f4b229fb4101 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:43.163299 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-42aec5e0-d486-4226-9de0-048732814dec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:43.163299 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:43.163299 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:43.163299 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:43.163299 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:43.163299 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:43.163299 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:43.163299 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:43.163299 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:43.163299 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:43.163299 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:43.163299 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:43.163299 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:43.163299 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:43.163299 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:43.163299 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:43.164161 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:43.164161 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:43.164161 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:43.164161 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:43.164161 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:43.164161 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:43.164161 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:43.164161 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:43.164161 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:43.164161 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:43.164161 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:43.164161 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:43.164161 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:43.164161 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:43.164161 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:43.164161 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:43.164161 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:43.164992 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:43.164992 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:43.164992 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:43.164992 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 193/402] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:43.165230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aec5e0-d486-4226-9de0-048732814dec None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:43.176925 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 210/403] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 07:15:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:43.177580 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a01cb-3aea-4783-9966-f4b229fb4101 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:43.275993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aec5e0-d486-4226-9de0-048732814dec None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:43.283110 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a01cb-3aea-4783-9966-f4b229fb4101 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:43.317746 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f383d73-6900-4db7-b906-814d84c0dfc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:43.318367 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f383d73-6900-4db7-b906-814d84c0dfc5 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:43.318706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f383d73-6900-4db7-b906-814d84c0dfc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:43.318955 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f383d73-6900-4db7-b906-814d84c0dfc5 None admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:43.319646 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f383d73-6900-4db7-b906-814d84c0dfc5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:43.319952 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f383d73-6900-4db7-b906-814d84c0dfc5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:43.321404 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f383d73-6900-4db7-b906-814d84c0dfc5 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:43.321507 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df18a2-88ac-4f3f-8e91-a66af7613c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:43.322214 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df18a2-88ac-4f3f-8e91-a66af7613c4b None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:43.322599 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df18a2-88ac-4f3f-8e91-a66af7613c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:43.323009 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df18a2-88ac-4f3f-8e91-a66af7613c4b None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:43.323685 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df18a2-88ac-4f3f-8e91-a66af7613c4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:43.324056 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df18a2-88ac-4f3f-8e91-a66af7613c4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:43.325486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df18a2-88ac-4f3f-8e91-a66af7613c4b None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:43.331175 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:43.334967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 211/404] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 07:15:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:43.335826 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f383d73-6900-4db7-b906-814d84c0dfc5 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:43.341774 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 194/405] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:43.346522 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df18a2-88ac-4f3f-8e91-a66af7613c4b None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:43.357505 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:43.443060 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f383d73-6900-4db7-b906-814d84c0dfc5 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:43.461240 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df18a2-88ac-4f3f-8e91-a66af7613c4b None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:43.478373 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecba2ae-300b-4aab-9a9a-77abcdba1a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:43.479162 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecba2ae-300b-4aab-9a9a-77abcdba1a0e None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:43.479505 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecba2ae-300b-4aab-9a9a-77abcdba1a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:43.479841 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecba2ae-300b-4aab-9a9a-77abcdba1a0e None admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:43.480548 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecba2ae-300b-4aab-9a9a-77abcdba1a0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:43.481008 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecba2ae-300b-4aab-9a9a-77abcdba1a0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:43.482499 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecba2ae-300b-4aab-9a9a-77abcdba1a0e None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:43.494686 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 212/406] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:43.495449 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecba2ae-300b-4aab-9a9a-77abcdba1a0e None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:43.501946 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944b3785-ebfb-4692-af8e-b1a819586fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:43.502614 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b3785-ebfb-4692-af8e-b1a819586fef None admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:43.502915 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b3785-ebfb-4692-af8e-b1a819586fef None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:43.503183 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b3785-ebfb-4692-af8e-b1a819586fef None admin] PATH_INFO: `/v3/projects/25e2bc8417754f909b416b93d5b7afb7/users/e6495385dccc4fa48c05b1feb957846c/roles/268fecadd98446f2b7d41755f3fe3649` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:43.518634 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944b3785-ebfb-4692-af8e-b1a819586fef None admin] RBAC: Authorizing `identity:create_grant(project_id=25e2bc8417754f909b416b93d5b7afb7, user_id=e6495385dccc4fa48c05b1feb957846c, role_id=268fecadd98446f2b7d41755f3fe3649)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:43.520122 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944b3785-ebfb-4692-af8e-b1a819586fef None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:43.535558 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 195/407] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 07:15:43 2022] PUT /identity/v3/projects/25e2bc8417754f909b416b93d5b7afb7/users/e6495385dccc4fa48c05b1feb957846c/roles/268fecadd98446f2b7d41755f3fe3649 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:43.536426 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944b3785-ebfb-4692-af8e-b1a819586fef None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:43.570407 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:43.618183 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:43.638086 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecba2ae-300b-4aab-9a9a-77abcdba1a0e None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:43.670915 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944b3785-ebfb-4692-af8e-b1a819586fef None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:43.686535 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c5871-4547-4383-97b4-06b91c9f9c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:43.687866 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c5871-4547-4383-97b4-06b91c9f9c12 None admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:43.688149 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c5871-4547-4383-97b4-06b91c9f9c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:43.688420 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c5871-4547-4383-97b4-06b91c9f9c12 None admin] PATH_INFO: `/v3/projects/25e2bc8417754f909b416b93d5b7afb7/users/9a051ca8154540ad93db6d028b8323a0/roles/268fecadd98446f2b7d41755f3fe3649` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:43.712165 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c5871-4547-4383-97b4-06b91c9f9c12 None admin] RBAC: Authorizing `identity:create_grant(project_id=25e2bc8417754f909b416b93d5b7afb7, user_id=9a051ca8154540ad93db6d028b8323a0, role_id=268fecadd98446f2b7d41755f3fe3649)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:43.713583 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c5871-4547-4383-97b4-06b91c9f9c12 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:43.720476 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a555f-dc2f-48fe-ac75-d9ba788d3838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:43.721824 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a555f-dc2f-48fe-ac75-d9ba788d3838 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:43.722261 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a555f-dc2f-48fe-ac75-d9ba788d3838 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:43.722748 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a555f-dc2f-48fe-ac75-d9ba788d3838 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:43.730927 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 213/408] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 07:15:43 2022] PUT /identity/v3/projects/25e2bc8417754f909b416b93d5b7afb7/users/9a051ca8154540ad93db6d028b8323a0/roles/268fecadd98446f2b7d41755f3fe3649 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:43.731962 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c5871-4547-4383-97b4-06b91c9f9c12 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:43.733506 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999a555f-dc2f-48fe-ac75-d9ba788d3838 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:43.735316 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999a555f-dc2f-48fe-ac75-d9ba788d3838 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:43.751205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-999a555f-dc2f-48fe-ac75-d9ba788d3838 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Nov 02 07:15:43.751205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:43.751205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:43.751205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:43.751205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:43.751205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:43.751205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:43.751205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:43.751205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:43.751205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:43.751205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:43.751205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:43.751205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:43.751205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:43.751205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:43.751205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:43.752197 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:43.752197 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:43.752197 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:43.752197 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:43.752197 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:43.752197 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:43.752197 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:43.752197 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:43.752197 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:43.752197 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:43.752197 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:43.752197 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:43.752197 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:43.752197 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:43.752197 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:43.752197 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:43.752197 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:43.752197 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:43.753078 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:43.753078 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:43.753078 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:43.753078 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:43.753078 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:43.753078 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:43.753078 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:43.753078 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:43.753078 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Nov 02 07:15:43.753078 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:43.755513 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 196/409] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Nov 2 07:15:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:43.762315 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a555f-dc2f-48fe-ac75-d9ba788d3838 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:43.851864 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c5871-4547-4383-97b4-06b91c9f9c12 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:43.879537 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a555f-dc2f-48fe-ac75-d9ba788d3838 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:43.881205 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:43.897884 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:43.922012 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c91708-5bbb-4823-9d77-636109407e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:43.922944 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c91708-5bbb-4823-9d77-636109407e06 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:43.923353 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c91708-5bbb-4823-9d77-636109407e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:43.923812 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c91708-5bbb-4823-9d77-636109407e06 None admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:43.924794 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c91708-5bbb-4823-9d77-636109407e06 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:43.925249 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c91708-5bbb-4823-9d77-636109407e06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:43.926886 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c91708-5bbb-4823-9d77-636109407e06 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:43.948314 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 197/410] 10.222.0.30 () {56 vars in 1139 bytes} [Wed Nov 2 07:15:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:43.949588 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1cdcdd-6f78-4c8f-81f5-327632cb2244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:43.950252 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1cdcdd-6f78-4c8f-81f5-327632cb2244 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:43.950528 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1cdcdd-6f78-4c8f-81f5-327632cb2244 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:43.954737 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1cdcdd-6f78-4c8f-81f5-327632cb2244 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:43.956128 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c91708-5bbb-4823-9d77-636109407e06 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:43.969738 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1cdcdd-6f78-4c8f-81f5-327632cb2244 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:43.972970 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1cdcdd-6f78-4c8f-81f5-327632cb2244 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:43.988973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-2d1cdcdd-6f78-4c8f-81f5-327632cb2244 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:43.988973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:43.988973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:43.988973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:43.988973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:43.988973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:43.988973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:43.988973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:43.988973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:43.988973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:43.988973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:43.988973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:43.988973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:15:43.988973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:15:43.988973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:15:43.990047 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:15:43.990047 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:43.990047 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:43.990047 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:43.990047 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:43.990047 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:43.990047 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:43.990047 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:43.990047 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:43.990047 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:43.990047 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:43.990047 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:43.990047 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:43.990047 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:43.990047 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:43.990047 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:43.990047 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:43.991975 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:15:43.991975 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:15:43.991975 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:15:43.991975 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:15:43.991975 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:15:43.991975 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:15:43.991975 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:43.991975 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:43.994056 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 214/411] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:43.999538 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1cdcdd-6f78-4c8f-81f5-327632cb2244 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:44.093182 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c91708-5bbb-4823-9d77-636109407e06 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:44.153908 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344efd9-5d49-47e5-9204-610070375880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:44.155527 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344efd9-5d49-47e5-9204-610070375880 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:44.156797 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344efd9-5d49-47e5-9204-610070375880 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:44.157327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344efd9-5d49-47e5-9204-610070375880 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:44.161217 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1cdcdd-6f78-4c8f-81f5-327632cb2244 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:44.180687 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4344efd9-5d49-47e5-9204-610070375880 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:44.182357 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4344efd9-5d49-47e5-9204-610070375880 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:44.205691 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-4344efd9-5d49-47e5-9204-610070375880 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:44.205691 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:44.205691 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:44.205691 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:44.205691 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:44.205691 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:44.205691 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:44.205691 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:44.205691 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:44.205691 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:44.205691 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:44.205691 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:44.205691 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:44.205691 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:44.205691 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:44.205691 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:44.206640 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:44.206640 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:44.206640 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:44.206640 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:44.206640 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:44.206640 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:44.206640 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:44.206640 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:44.206640 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:44.206640 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:44.206640 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:44.206640 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:44.206640 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:44.206640 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:44.206640 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:44.206640 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:44.206640 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:44.207725 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:44.207725 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:44.207725 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:44.207725 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 198/412] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:44.212606 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344efd9-5d49-47e5-9204-610070375880 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:44.213409 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca7580-b429-4e5c-ab6e-e25f585e7e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:44.214145 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca7580-b429-4e5c-ab6e-e25f585e7e6a None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:44.214537 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca7580-b429-4e5c-ab6e-e25f585e7e6a None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:44.214956 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca7580-b429-4e5c-ab6e-e25f585e7e6a None admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:44.215843 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca7580-b429-4e5c-ab6e-e25f585e7e6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:44.216336 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca7580-b429-4e5c-ab6e-e25f585e7e6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:44.218233 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca7580-b429-4e5c-ab6e-e25f585e7e6a None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:44.243328 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:44.246293 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 215/413] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 07:15:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:44.261388 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca7580-b429-4e5c-ab6e-e25f585e7e6a None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:44.319029 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:44.323516 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344efd9-5d49-47e5-9204-610070375880 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:44.357672 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6faa93-f2ac-4de8-89e8-92ba0feddde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:44.358262 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6faa93-f2ac-4de8-89e8-92ba0feddde3 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:44.358533 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6faa93-f2ac-4de8-89e8-92ba0feddde3 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:44.358828 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6faa93-f2ac-4de8-89e8-92ba0feddde3 None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:44.359466 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6faa93-f2ac-4de8-89e8-92ba0feddde3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:44.359760 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6faa93-f2ac-4de8-89e8-92ba0feddde3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:44.361147 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6faa93-f2ac-4de8-89e8-92ba0feddde3 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:44.373835 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 199/414] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:44.379309 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6faa93-f2ac-4de8-89e8-92ba0feddde3 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:44.452887 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca7580-b429-4e5c-ab6e-e25f585e7e6a None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:44.471109 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:44.479539 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6faa93-f2ac-4de8-89e8-92ba0feddde3 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:44.517887 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbd3b6a-bbc0-4524-b1e6-22087dde7a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:44.518485 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbd3b6a-bbc0-4524-b1e6-22087dde7a5e None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:44.518786 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbd3b6a-bbc0-4524-b1e6-22087dde7a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:44.519052 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbd3b6a-bbc0-4524-b1e6-22087dde7a5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:44.530237 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbd3b6a-bbc0-4524-b1e6-22087dde7a5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:44.532085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbd3b6a-bbc0-4524-b1e6-22087dde7a5e None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:44.546261 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da35015-b330-480e-a372-a190709c718b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:44.547096 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da35015-b330-480e-a372-a190709c718b None admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:44.547424 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da35015-b330-480e-a372-a190709c718b None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:44.547695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da35015-b330-480e-a372-a190709c718b None admin] PATH_INFO: `/v3/projects/25e2bc8417754f909b416b93d5b7afb7/users/315861510b1041dca19d1e8753d9b21d/roles/268fecadd98446f2b7d41755f3fe3649` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:44.552911 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-bdbd3b6a-bbc0-4524-b1e6-22087dde7a5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:44.552911 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:44.552911 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:44.552911 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:44.552911 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:44.552911 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:44.552911 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:44.552911 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:44.552911 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:44.552911 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:44.552911 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:44.552911 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:44.552911 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:15:44.552911 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:15:44.552911 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:15:44.553723 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:15:44.553723 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:44.553723 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:44.553723 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:44.553723 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:44.553723 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:44.553723 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:44.553723 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:44.553723 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:44.553723 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:44.553723 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:44.553723 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:44.553723 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:44.553723 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:44.553723 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:44.553723 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:44.553723 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:44.554996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:15:44.554996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:15:44.554996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:15:44.554996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:15:44.554996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:15:44.554996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:15:44.554996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:44.554996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:44.554996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 200/415] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:44.560214 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbd3b6a-bbc0-4524-b1e6-22087dde7a5e None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:44.568597 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da35015-b330-480e-a372-a190709c718b None admin] RBAC: Authorizing `identity:create_grant(project_id=25e2bc8417754f909b416b93d5b7afb7, user_id=315861510b1041dca19d1e8753d9b21d, role_id=268fecadd98446f2b7d41755f3fe3649)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:44.571280 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da35015-b330-480e-a372-a190709c718b None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:44.594947 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 216/416] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 07:15:44 2022] PUT /identity/v3/projects/25e2bc8417754f909b416b93d5b7afb7/users/315861510b1041dca19d1e8753d9b21d/roles/268fecadd98446f2b7d41755f3fe3649 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:15:44.709803 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbd3b6a-bbc0-4524-b1e6-22087dde7a5e None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:44.746010 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f7bd7-15ed-48be-8f62-642d8309a673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:44.746702 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f7bd7-15ed-48be-8f62-642d8309a673 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:44.746964 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f7bd7-15ed-48be-8f62-642d8309a673 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:44.747223 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f7bd7-15ed-48be-8f62-642d8309a673 None admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:44.747974 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2f7bd7-15ed-48be-8f62-642d8309a673 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:44.748498 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2f7bd7-15ed-48be-8f62-642d8309a673 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:44.750043 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2f7bd7-15ed-48be-8f62-642d8309a673 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:44.769875 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 201/417] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 07:15:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:44.775924 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da35015-b330-480e-a372-a190709c718b None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:44.895717 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:44.919789 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da35015-b330-480e-a372-a190709c718b None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:44.956502 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ad1b4-4912-4574-9af6-ee30e4d0fac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:44.957169 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ad1b4-4912-4574-9af6-ee30e4d0fac2 None admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:44.957445 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ad1b4-4912-4574-9af6-ee30e4d0fac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:44.957716 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ad1b4-4912-4574-9af6-ee30e4d0fac2 None admin] PATH_INFO: `/v3/projects/25e2bc8417754f909b416b93d5b7afb7/users/65d51ec74e1e4c25bcc4365eb858bac0/roles/268fecadd98446f2b7d41755f3fe3649` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:44.973424 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ad1b4-4912-4574-9af6-ee30e4d0fac2 None admin] RBAC: Authorizing `identity:create_grant(project_id=25e2bc8417754f909b416b93d5b7afb7, user_id=65d51ec74e1e4c25bcc4365eb858bac0, role_id=268fecadd98446f2b7d41755f3fe3649)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:44.973617 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c090a14-7090-4fe2-8d5c-17455d6b57b6 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:44.973988 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c090a14-7090-4fe2-8d5c-17455d6b57b6 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:44.974379 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c090a14-7090-4fe2-8d5c-17455d6b57b6 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:44.974979 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ad1b4-4912-4574-9af6-ee30e4d0fac2 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:44.976127 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 202/418] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:44.981438 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb03d61-6750-4a6c-b902-3ef2456b3a4f None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:44.981774 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb03d61-6750-4a6c-b902-3ef2456b3a4f None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:44.982164 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb03d61-6750-4a6c-b902-3ef2456b3a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:44.993669 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 217/419] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 07:15:44 2022] PUT /identity/v3/projects/25e2bc8417754f909b416b93d5b7afb7/users/65d51ec74e1e4c25bcc4365eb858bac0/roles/268fecadd98446f2b7d41755f3fe3649 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:45.007253 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:45.015123 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-1eb03d61-6750-4a6c-b902-3ef2456b3a4f None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:45.017246 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-1eb03d61-6750-4a6c-b902-3ef2456b3a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:45.113139 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 203/420] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:15:45.465472 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ad1b4-4912-4574-9af6-ee30e4d0fac2 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:45.589447 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ad1b4-4912-4574-9af6-ee30e4d0fac2 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:45.646199 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:45.657551 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388e111-2586-4b56-9f17-bf6fbb8976c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:45.658485 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388e111-2586-4b56-9f17-bf6fbb8976c1 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:45.658990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388e111-2586-4b56-9f17-bf6fbb8976c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:45.659508 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388e111-2586-4b56-9f17-bf6fbb8976c1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:45.677910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4388e111-2586-4b56-9f17-bf6fbb8976c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:45.683115 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-4388e111-2586-4b56-9f17-bf6fbb8976c1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:45.683115 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:45.683115 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:45.683115 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:45.683115 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:45.683115 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:45.683115 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:45.683115 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:45.683115 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:45.683115 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:45.683115 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:45.683115 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:45.683115 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:45.683115 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:45.683115 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:45.683115 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:45.683982 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:45.683982 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:45.683982 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:45.683982 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:45.683982 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:45.683982 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:45.683982 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:45.683982 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:45.683982 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:45.683982 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:45.683982 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:45.683982 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:45.683982 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:45.683982 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:45.683982 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:45.683982 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:45.684839 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:45.684839 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:45.684839 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:45.684839 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:45.684839 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:45.684839 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:45.684839 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:45.684839 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:45.684839 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:45.684839 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:45.684839 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:45.684839 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:45.687593 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 218/421] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Nov 2 07:15:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:45.693116 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb03d61-6750-4a6c-b902-3ef2456b3a4f None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:45.815365 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb03d61-6750-4a6c-b902-3ef2456b3a4f None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:45.855121 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71764619-1d72-45ec-9fe5-8ef0a5866659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:45.856086 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71764619-1d72-45ec-9fe5-8ef0a5866659 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:45.856557 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71764619-1d72-45ec-9fe5-8ef0a5866659 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:45.857058 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71764619-1d72-45ec-9fe5-8ef0a5866659 None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:45.857996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71764619-1d72-45ec-9fe5-8ef0a5866659 None admin] RBAC: Adding query filter params (name=service) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:45.858501 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71764619-1d72-45ec-9fe5-8ef0a5866659 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:45.860101 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71764619-1d72-45ec-9fe5-8ef0a5866659 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:45.862774 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:45.870591 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 204/422] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Nov 2 07:15:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:45.875942 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388e111-2586-4b56-9f17-bf6fbb8976c1 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:45.994357 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388e111-2586-4b56-9f17-bf6fbb8976c1 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:46.030266 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039900dc-dafd-43f0-949b-6068753027bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:46.031311 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039900dc-dafd-43f0-949b-6068753027bb None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:46.032158 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039900dc-dafd-43f0-949b-6068753027bb None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:46.032622 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039900dc-dafd-43f0-949b-6068753027bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:46.043488 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039900dc-dafd-43f0-949b-6068753027bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:46.045279 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039900dc-dafd-43f0-949b-6068753027bb None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:46.060200 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-039900dc-dafd-43f0-949b-6068753027bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:46.060200 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:46.060200 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:46.060200 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:46.060200 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:46.060200 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:46.060200 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:46.060200 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:46.060200 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:46.060200 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:46.060200 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:46.060200 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:46.060200 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:46.060200 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:46.060200 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:46.060200 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:46.061217 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:46.061217 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:46.061217 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:46.061217 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:46.061217 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:46.061217 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:46.061217 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:46.061217 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:46.061217 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:46.061217 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:46.061217 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:46.061217 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:46.061217 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:46.061217 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:46.061217 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:46.061217 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:46.061217 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:46.062157 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:46.062157 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:46.062157 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:46.064260 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 219/423] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:46.071598 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71764619-1d72-45ec-9fe5-8ef0a5866659 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:46.206351 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71764619-1d72-45ec-9fe5-8ef0a5866659 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:46.248826 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656065a8-d80d-4970-962b-eec75a2ae9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:46.249815 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656065a8-d80d-4970-962b-eec75a2ae9bb None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:46.250290 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656065a8-d80d-4970-962b-eec75a2ae9bb None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:46.250784 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656065a8-d80d-4970-962b-eec75a2ae9bb None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:46.251637 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656065a8-d80d-4970-962b-eec75a2ae9bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:46.252154 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656065a8-d80d-4970-962b-eec75a2ae9bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:46.253735 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656065a8-d80d-4970-962b-eec75a2ae9bb None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:46.272452 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 205/424] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:46.276353 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a235b-b210-4e89-bd79-e9a9620059cc None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:46.276625 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a235b-b210-4e89-bd79-e9a9620059cc None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:46.276909 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a235b-b210-4e89-bd79-e9a9620059cc None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:46.277958 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656065a8-d80d-4970-962b-eec75a2ae9bb None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:46.278532 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 220/425] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:46.284769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2f7c3-6387-40fb-a7d2-442f37d7b09e None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:46.285076 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2f7c3-6387-40fb-a7d2-442f37d7b09e None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:46.285305 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2f7c3-6387-40fb-a7d2-442f37d7b09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:46.308607 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-aff2f7c3-6387-40fb-a7d2-442f37d7b09e None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:46.310874 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-aff2f7c3-6387-40fb-a7d2-442f37d7b09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:46.375697 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:46.379584 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:46.439135 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 221/426] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:15:46.440490 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e7b48-3b77-4f78-8011-1df1b0fa0ae1 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:46.440774 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e7b48-3b77-4f78-8011-1df1b0fa0ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:46.441047 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e7b48-3b77-4f78-8011-1df1b0fa0ae1 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:46.442525 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 222/427] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:46.443863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656065a8-d80d-4970-962b-eec75a2ae9bb None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:46.448078 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7de38-b8cd-4065-b7c0-7695469126fd None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:46.448359 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7de38-b8cd-4065-b7c0-7695469126fd None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:46.448628 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7de38-b8cd-4065-b7c0-7695469126fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:46.480807 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300688bd-d309-43ac-b49b-01db7a4bcbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:46.481482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300688bd-d309-43ac-b49b-01db7a4bcbe3 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:46.481754 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300688bd-d309-43ac-b49b-01db7a4bcbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:46.482018 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300688bd-d309-43ac-b49b-01db7a4bcbe3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:46.487780 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-6ef7de38-b8cd-4065-b7c0-7695469126fd None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:46.488987 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-6ef7de38-b8cd-4065-b7c0-7695469126fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:46.489792 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300688bd-d309-43ac-b49b-01db7a4bcbe3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:46.491317 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300688bd-d309-43ac-b49b-01db7a4bcbe3 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:46.503104 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-300688bd-d309-43ac-b49b-01db7a4bcbe3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Nov 02 07:15:46.503104 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:46.503104 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:46.503104 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:46.503104 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:46.503104 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:46.503104 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:46.503104 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:46.503104 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:46.503104 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:46.503104 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:46.503104 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:46.503104 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:46.503104 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:46.503104 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:46.503104 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:46.504001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:46.504001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:46.504001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:46.504001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:46.504001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:46.504001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:46.504001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:46.504001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:46.504001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:46.504001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:46.504001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:46.504001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:46.504001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:46.504001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:46.504001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:46.504001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:46.504001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:46.504001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:46.504880 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:46.504880 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:46.504880 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:46.504880 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:46.504880 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:46.504880 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:46.504880 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:46.504880 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:46.504880 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Nov 02 07:15:46.504880 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:46.504880 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 206/428] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 07:15:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:46.510045 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300688bd-d309-43ac-b49b-01db7a4bcbe3 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:46.615116 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300688bd-d309-43ac-b49b-01db7a4bcbe3 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:46.632814 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:46.650570 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af85ae-9fdf-4de4-b070-4f75b9d7ab6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:46.651336 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af85ae-9fdf-4de4-b070-4f75b9d7ab6c None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:46.651616 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af85ae-9fdf-4de4-b070-4f75b9d7ab6c None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:46.651883 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af85ae-9fdf-4de4-b070-4f75b9d7ab6c None admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:46.652604 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af85ae-9fdf-4de4-b070-4f75b9d7ab6c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:46.652902 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af85ae-9fdf-4de4-b070-4f75b9d7ab6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:46.654538 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af85ae-9fdf-4de4-b070-4f75b9d7ab6c None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:46.673305 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 207/429] 10.222.0.30 () {56 vars in 1135 bytes} [Wed Nov 2 07:15:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:46.678978 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af85ae-9fdf-4de4-b070-4f75b9d7ab6c None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:46.716295 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 223/430] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:46.717265 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7de38-b8cd-4065-b7c0-7695469126fd None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:46.787871 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af85ae-9fdf-4de4-b070-4f75b9d7ab6c None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:46.846071 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eed731-26e9-458d-89d0-7e004082bd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:46.846781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eed731-26e9-458d-89d0-7e004082bd4c None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:46.847198 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eed731-26e9-458d-89d0-7e004082bd4c None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:46.848594 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eed731-26e9-458d-89d0-7e004082bd4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:46.873033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eed731-26e9-458d-89d0-7e004082bd4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:46.875245 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eed731-26e9-458d-89d0-7e004082bd4c None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:46.907105 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7de38-b8cd-4065-b7c0-7695469126fd None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:46.908526 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:46.910536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-71eed731-26e9-458d-89d0-7e004082bd4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:46.910536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:46.910536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:46.910536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:46.910536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:46.910536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:46.910536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:46.910536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:46.910536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:46.910536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:46.910536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:46.910536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:46.910536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:46.910536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:46.910536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:46.910536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:46.911472 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:46.911472 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:46.911472 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:46.911472 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:46.911472 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:46.911472 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:46.911472 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:46.911472 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:46.911472 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:46.911472 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:46.911472 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:46.911472 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:46.911472 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:46.911472 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:46.911472 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:46.911472 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:46.911472 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:46.912282 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:46.912282 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:46.912282 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:46.912282 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 208/431] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:46.925063 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eed731-26e9-458d-89d0-7e004082bd4c None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:46.958927 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:46.976148 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c37b9a3-ceb8-4ea1-8692-266205a30dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:46.976792 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37b9a3-ceb8-4ea1-8692-266205a30dab None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:46.977065 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37b9a3-ceb8-4ea1-8692-266205a30dab None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:46.977329 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37b9a3-ceb8-4ea1-8692-266205a30dab None admin] PATH_INFO: `/v3/roles/service` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:46.991152 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c37b9a3-ceb8-4ea1-8692-266205a30dab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:46.996648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-8c37b9a3-ceb8-4ea1-8692-266205a30dab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:46.996648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:46.996648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:46.996648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:46.996648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:46.996648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:46.996648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:46.996648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:46.996648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:46.996648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:46.996648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:46.996648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:46.996648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:46.996648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:46.996648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:46.996648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:46.997594 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:46.997594 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:46.997594 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:46.997594 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:46.997594 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:46.997594 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:46.997594 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:46.997594 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:46.997594 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:46.997594 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:46.997594 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:46.997594 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:46.997594 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:46.997594 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:46.997594 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:46.997594 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:46.998407 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:46.998407 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:46.998407 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:46.998407 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:46.998407 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:46.998407 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:46.998407 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:46.998407 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:46.998407 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:46.998407 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:46.998407 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:46.998407 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:47.000734 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 224/432] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Nov 2 07:15:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:47.011532 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c37b9a3-ceb8-4ea1-8692-266205a30dab None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:47.049269 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eed731-26e9-458d-89d0-7e004082bd4c None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:47.093623 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49bad97-0efe-4db3-a0dc-502da169eb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:47.094244 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49bad97-0efe-4db3-a0dc-502da169eb27 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:47.094516 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49bad97-0efe-4db3-a0dc-502da169eb27 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:47.096617 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49bad97-0efe-4db3-a0dc-502da169eb27 None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:47.097280 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49bad97-0efe-4db3-a0dc-502da169eb27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:47.097585 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49bad97-0efe-4db3-a0dc-502da169eb27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:47.099010 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49bad97-0efe-4db3-a0dc-502da169eb27 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:47.115097 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 209/433] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:47.116323 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49bad97-0efe-4db3-a0dc-502da169eb27 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:47.161973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c37b9a3-ceb8-4ea1-8692-266205a30dab None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:47.198453 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1996174e-7849-49d9-8277-30890f243706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:47.199110 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996174e-7849-49d9-8277-30890f243706 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:47.199388 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996174e-7849-49d9-8277-30890f243706 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:47.199657 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996174e-7849-49d9-8277-30890f243706 None admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:47.200318 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1996174e-7849-49d9-8277-30890f243706 None admin] RBAC: Adding query filter params (name=service) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:47.200616 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1996174e-7849-49d9-8277-30890f243706 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:47.202100 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1996174e-7849-49d9-8277-30890f243706 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:47.211397 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 225/434] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Nov 2 07:15:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:47.212240 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1996174e-7849-49d9-8277-30890f243706 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:47.223637 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:47.239439 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49bad97-0efe-4db3-a0dc-502da169eb27 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:47.244555 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:47.281729 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363efba7-1a98-475f-bf65-3fbb7d828a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:47.282982 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363efba7-1a98-475f-bf65-3fbb7d828a5c None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:47.283857 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363efba7-1a98-475f-bf65-3fbb7d828a5c None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:47.284572 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363efba7-1a98-475f-bf65-3fbb7d828a5c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:47.296730 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363efba7-1a98-475f-bf65-3fbb7d828a5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:47.303584 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-363efba7-1a98-475f-bf65-3fbb7d828a5c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:47.303584 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:47.303584 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:47.303584 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:47.303584 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:47.303584 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:47.303584 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:47.303584 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:47.303584 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:47.303584 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:47.303584 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:47.303584 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:47.303584 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:47.303584 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:47.303584 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:47.303584 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:47.304483 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:47.304483 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:47.304483 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:47.304483 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:47.304483 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:47.304483 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:47.304483 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:47.304483 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:47.304483 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:47.304483 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:47.304483 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:47.304483 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:47.304483 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:47.304483 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:47.304483 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:47.304483 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:47.305252 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:47.305252 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:47.305252 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:47.305252 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:47.305252 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:47.305252 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:47.305252 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:47.305252 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:47.305252 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:47.305252 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:47.305252 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:47.305252 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:47.312693 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 210/435] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Nov 2 07:15:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:47.313336 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363efba7-1a98-475f-bf65-3fbb7d828a5c None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:47.330090 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1996174e-7849-49d9-8277-30890f243706 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:47.367368 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9777729-a5c4-4f04-8bc4-0d6d9165f01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:47.368053 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9777729-a5c4-4f04-8bc4-0d6d9165f01b None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:47.368325 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9777729-a5c4-4f04-8bc4-0d6d9165f01b None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:47.368657 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9777729-a5c4-4f04-8bc4-0d6d9165f01b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:47.376437 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9777729-a5c4-4f04-8bc4-0d6d9165f01b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:47.378135 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9777729-a5c4-4f04-8bc4-0d6d9165f01b None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:47.390588 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-a9777729-a5c4-4f04-8bc4-0d6d9165f01b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:47.390588 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:47.390588 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:47.390588 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:47.390588 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:47.390588 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:47.390588 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:47.390588 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:47.390588 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:47.390588 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:47.390588 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:47.390588 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:47.390588 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:15:47.390588 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:15:47.390588 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:15:47.391507 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:15:47.391507 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:47.391507 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:47.391507 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:47.391507 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:47.391507 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:47.391507 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:47.391507 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:47.391507 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:47.391507 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:47.391507 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:47.391507 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:47.391507 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:47.391507 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:47.391507 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:47.391507 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:47.391507 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:47.392327 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:15:47.392327 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:15:47.392327 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:15:47.392327 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:15:47.392327 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:15:47.392327 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:15:47.392327 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:47.392327 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:47.392327 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 226/436] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:47.393265 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9777729-a5c4-4f04-8bc4-0d6d9165f01b None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:47.437942 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363efba7-1a98-475f-bf65-3fbb7d828a5c None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:47.480577 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951e8c9-1bef-479e-98d3-01afff95a75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:47.481911 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951e8c9-1bef-479e-98d3-01afff95a75e None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:47.482415 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951e8c9-1bef-479e-98d3-01afff95a75e None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:47.483542 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951e8c9-1bef-479e-98d3-01afff95a75e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:47.486070 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:47.499825 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d951e8c9-1bef-479e-98d3-01afff95a75e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:47.502458 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d951e8c9-1bef-479e-98d3-01afff95a75e None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:47.509263 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9777729-a5c4-4f04-8bc4-0d6d9165f01b None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:47.512701 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:47.521230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-d951e8c9-1bef-479e-98d3-01afff95a75e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:47.521230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:47.521230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:47.521230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:47.521230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:47.521230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:47.521230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:47.521230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:47.521230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:47.521230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:47.521230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:47.521230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:47.521230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:47.521230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:47.521230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:47.521230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:47.522347 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:47.522347 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:47.522347 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:47.522347 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:47.522347 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:47.522347 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:47.522347 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:47.522347 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:47.522347 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:47.522347 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:47.522347 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:47.522347 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:47.522347 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:47.522347 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:47.522347 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:47.522347 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:47.522347 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:47.523429 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:47.523429 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:47.523429 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:47.526726 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 211/437] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:47.527692 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951e8c9-1bef-479e-98d3-01afff95a75e None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:47.560778 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d90384-ccd1-4ea6-9e25-6080e62eb689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:47.561983 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d90384-ccd1-4ea6-9e25-6080e62eb689 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:47.562354 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d90384-ccd1-4ea6-9e25-6080e62eb689 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:47.562745 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d90384-ccd1-4ea6-9e25-6080e62eb689 None admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:47.563687 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d90384-ccd1-4ea6-9e25-6080e62eb689 None admin] RBAC: Adding query filter params (name=service) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:47.564094 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d90384-ccd1-4ea6-9e25-6080e62eb689 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:47.565783 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d90384-ccd1-4ea6-9e25-6080e62eb689 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:47.581073 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 227/438] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Nov 2 07:15:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:47.581073 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d90384-ccd1-4ea6-9e25-6080e62eb689 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:47.654028 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951e8c9-1bef-479e-98d3-01afff95a75e None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:47.686764 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d90384-ccd1-4ea6-9e25-6080e62eb689 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:47.688557 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7629d277-1a81-4bd0-9536-9ee1236ec6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:47.689120 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629d277-1a81-4bd0-9536-9ee1236ec6e7 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:47.689394 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629d277-1a81-4bd0-9536-9ee1236ec6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:47.689662 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629d277-1a81-4bd0-9536-9ee1236ec6e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:47.690348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7629d277-1a81-4bd0-9536-9ee1236ec6e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:47.690670 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7629d277-1a81-4bd0-9536-9ee1236ec6e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:47.692104 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7629d277-1a81-4bd0-9536-9ee1236ec6e7 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:47.704347 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 212/439] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 07:15:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:47.705136 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7629d277-1a81-4bd0-9536-9ee1236ec6e7 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:47.719500 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0d8b4-9b01-496a-8dee-1e212e3c722f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:47.720104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0d8b4-9b01-496a-8dee-1e212e3c722f None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:47.720375 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0d8b4-9b01-496a-8dee-1e212e3c722f None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:47.720666 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0d8b4-9b01-496a-8dee-1e212e3c722f None admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:47.721294 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0d8b4-9b01-496a-8dee-1e212e3c722f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:47.721584 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0d8b4-9b01-496a-8dee-1e212e3c722f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:47.722977 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0d8b4-9b01-496a-8dee-1e212e3c722f None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:47.734353 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 228/440] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:47.735558 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701460d2-25e8-4b32-bfa5-5b8204db13fc None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:47.735832 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701460d2-25e8-4b32-bfa5-5b8204db13fc None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:47.736097 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701460d2-25e8-4b32-bfa5-5b8204db13fc None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:47.737280 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 229/441] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:47.738254 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5991e-b077-43c6-8fd5-978ef0d89128 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:47.738525 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5991e-b077-43c6-8fd5-978ef0d89128 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:47.738810 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5991e-b077-43c6-8fd5-978ef0d89128 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:47.739945 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 230/442] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:47.740605 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5991e-b077-43c6-8fd5-978ef0d89128 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:47.750007 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:47.771798 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:47.812620 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7629d277-1a81-4bd0-9536-9ee1236ec6e7 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:47.844687 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5991e-b077-43c6-8fd5-978ef0d89128 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:47.846901 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d23ad-63ad-4a04-bdb2-bad103b35003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:47.847535 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d23ad-63ad-4a04-bdb2-bad103b35003 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:47.847822 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d23ad-63ad-4a04-bdb2-bad103b35003 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:47.848084 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d23ad-63ad-4a04-bdb2-bad103b35003 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:47.857962 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0d23ad-63ad-4a04-bdb2-bad103b35003 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:47.859476 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0d23ad-63ad-4a04-bdb2-bad103b35003 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:47.874757 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-ad0d23ad-63ad-4a04-bdb2-bad103b35003 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:47.874757 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:47.874757 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:47.874757 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:47.874757 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:47.874757 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:47.874757 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:47.874757 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:47.874757 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:47.874757 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:47.874757 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:47.874757 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:47.874757 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:47.874757 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:47.874757 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:47.874757 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:47.875660 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:47.875660 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:47.875660 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:47.875660 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:47.875660 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:47.875660 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:47.875660 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:47.875660 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:47.875660 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:47.875660 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:47.875660 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:47.875660 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:47.875660 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:47.875660 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:47.875660 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:47.875660 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:47.875660 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:47.876480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:47.876480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:47.876480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:47.876480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 213/443] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:47.879518 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d23ad-63ad-4a04-bdb2-bad103b35003 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:47.880906 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e73cc2b-aae3-4acb-ab88-bd56464b8b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:47.881563 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73cc2b-aae3-4acb-ab88-bd56464b8b58 None admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:47.881838 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73cc2b-aae3-4acb-ab88-bd56464b8b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:47.882126 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73cc2b-aae3-4acb-ab88-bd56464b8b58 None admin] PATH_INFO: `/v3/projects/25e2bc8417754f909b416b93d5b7afb7/users/836c4044aaf74de391219289b57d2956/roles/268fecadd98446f2b7d41755f3fe3649` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:47.896910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e73cc2b-aae3-4acb-ab88-bd56464b8b58 None admin] RBAC: Authorizing `identity:create_grant(project_id=25e2bc8417754f909b416b93d5b7afb7, user_id=836c4044aaf74de391219289b57d2956, role_id=268fecadd98446f2b7d41755f3fe3649)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:47.898357 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e73cc2b-aae3-4acb-ab88-bd56464b8b58 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:47.912280 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 231/444] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 07:15:47 2022] PUT /identity/v3/projects/25e2bc8417754f909b416b93d5b7afb7/users/836c4044aaf74de391219289b57d2956/roles/268fecadd98446f2b7d41755f3fe3649 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:47.913469 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8c4d0-2750-4d2f-9132-1a488ac6551a None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:47.913769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8c4d0-2750-4d2f-9132-1a488ac6551a None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:47.914017 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8c4d0-2750-4d2f-9132-1a488ac6551a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:47.942993 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-93e8c4d0-2750-4d2f-9132-1a488ac6551a None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:47.944489 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-93e8c4d0-2750-4d2f-9132-1a488ac6551a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:47.985708 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d23ad-63ad-4a04-bdb2-bad103b35003 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:47.999568 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:48.004955 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:48.023113 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639267e2-a98e-4bb2-aa3d-4b7b9a550514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:48.023753 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639267e2-a98e-4bb2-aa3d-4b7b9a550514 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:48.024026 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639267e2-a98e-4bb2-aa3d-4b7b9a550514 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:48.024295 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639267e2-a98e-4bb2-aa3d-4b7b9a550514 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:48.035009 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 232/445] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:48.035009 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6419491e-4fcd-412e-aad7-fed0517bf9fe None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:48.035009 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6419491e-4fcd-412e-aad7-fed0517bf9fe None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:48.035009 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6419491e-4fcd-412e-aad7-fed0517bf9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:48.043103 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639267e2-a98e-4bb2-aa3d-4b7b9a550514 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:48.044650 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639267e2-a98e-4bb2-aa3d-4b7b9a550514 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:48.051809 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-6419491e-4fcd-412e-aad7-fed0517bf9fe None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:48.053026 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-6419491e-4fcd-412e-aad7-fed0517bf9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:48.057818 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-639267e2-a98e-4bb2-aa3d-4b7b9a550514 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:15:48.057818 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:48.057818 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:48.057818 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:48.057818 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:48.057818 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:48.057818 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:48.057818 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:48.057818 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:48.057818 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:48.057818 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:48.057818 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:48.057818 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:48.057818 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:48.057818 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:48.057818 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:48.058787 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:48.058787 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:48.058787 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:48.058787 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:48.058787 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:48.058787 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:48.058787 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:48.058787 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:48.058787 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:48.058787 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:48.058787 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:48.058787 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:48.058787 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:48.058787 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:48.058787 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:48.058787 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:48.058787 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:48.058787 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:48.059668 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:48.059668 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:48.059668 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:48.059668 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:48.059668 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:48.059668 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:48.059668 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:48.059668 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:48.059668 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:15:48.059668 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:48.059668 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 214/446] 10.222.0.30 () {56 vars in 1093 bytes} [Wed Nov 2 07:15:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:48.060809 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639267e2-a98e-4bb2-aa3d-4b7b9a550514 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:48.143646 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 233/447] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:48.144524 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6419491e-4fcd-412e-aad7-fed0517bf9fe None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:48.194355 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639267e2-a98e-4bb2-aa3d-4b7b9a550514 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:48.258821 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe1da9-4c0d-405d-a98c-fe5f85ca6da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:48.259435 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe1da9-4c0d-405d-a98c-fe5f85ca6da2 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:48.259709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe1da9-4c0d-405d-a98c-fe5f85ca6da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:48.259978 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe1da9-4c0d-405d-a98c-fe5f85ca6da2 None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:48.260622 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe1da9-4c0d-405d-a98c-fe5f85ca6da2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:48.260922 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe1da9-4c0d-405d-a98c-fe5f85ca6da2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:48.262301 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe1da9-4c0d-405d-a98c-fe5f85ca6da2 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:48.276129 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 215/448] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:48.282517 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe1da9-4c0d-405d-a98c-fe5f85ca6da2 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:48.315015 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:48.347178 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6419491e-4fcd-412e-aad7-fed0517bf9fe None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:48.366472 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:48.386823 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fae145-fe7d-4241-a7cb-b332d9cc16c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:48.387580 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fae145-fe7d-4241-a7cb-b332d9cc16c0 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:48.387858 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fae145-fe7d-4241-a7cb-b332d9cc16c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:48.388125 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fae145-fe7d-4241-a7cb-b332d9cc16c0 None admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:48.388925 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fae145-fe7d-4241-a7cb-b332d9cc16c0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:48.389275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fae145-fe7d-4241-a7cb-b332d9cc16c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:48.390977 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fae145-fe7d-4241-a7cb-b332d9cc16c0 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:48.393196 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe1da9-4c0d-405d-a98c-fe5f85ca6da2 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:48.410235 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 234/449] 10.222.0.30 () {56 vars in 1133 bytes} [Wed Nov 2 07:15:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:48.411208 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fae145-fe7d-4241-a7cb-b332d9cc16c0 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:48.430526 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc916a-1a18-4693-be72-748a514d1062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:48.431283 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc916a-1a18-4693-be72-748a514d1062 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:48.431600 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc916a-1a18-4693-be72-748a514d1062 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:48.431918 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc916a-1a18-4693-be72-748a514d1062 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:48.440529 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc916a-1a18-4693-be72-748a514d1062 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:48.442125 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc916a-1a18-4693-be72-748a514d1062 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:48.455312 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-8cdc916a-1a18-4693-be72-748a514d1062 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Nov 02 07:15:48.455312 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:48.455312 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:48.455312 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:48.455312 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:48.455312 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:48.455312 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:48.455312 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:48.455312 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:48.455312 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:48.455312 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:48.455312 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:48.455312 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:48.455312 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:48.455312 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:48.455312 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:48.456718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:48.456718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:48.456718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:48.456718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:48.456718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:48.456718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:48.456718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:48.456718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:48.456718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:48.456718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:48.456718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:48.456718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:48.456718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:48.456718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:48.456718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:48.456718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:48.456718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:48.456718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:48.459033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:48.459033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:48.459033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:48.459033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:48.459033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:48.459033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:48.459033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:48.459033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:48.459033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Nov 02 07:15:48.459033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:48.459033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 216/450] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:48.459033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc916a-1a18-4693-be72-748a514d1062 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:48.522820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fae145-fe7d-4241-a7cb-b332d9cc16c0 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:48.550079 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:48.560161 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc916a-1a18-4693-be72-748a514d1062 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:48.565564 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b4e4a-0022-4cd9-9450-4a8078faaabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:48.566195 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b4e4a-0022-4cd9-9450-4a8078faaabf None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:48.566469 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b4e4a-0022-4cd9-9450-4a8078faaabf None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:48.566856 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b4e4a-0022-4cd9-9450-4a8078faaabf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:48.577269 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635b4e4a-0022-4cd9-9450-4a8078faaabf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:48.582820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-635b4e4a-0022-4cd9-9450-4a8078faaabf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:48.582820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:48.582820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:48.582820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:48.582820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:48.582820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:48.582820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:48.582820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:48.582820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:48.582820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:48.582820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:48.582820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:48.582820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:48.582820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:48.582820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:48.582820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:48.583689 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:48.583689 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:48.583689 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:48.583689 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:48.583689 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:48.583689 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:48.583689 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:48.583689 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:48.583689 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:48.583689 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:48.583689 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:48.583689 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:48.583689 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:48.583689 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:48.583689 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:48.583689 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:48.584566 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:48.584566 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:48.584566 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:48.584566 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:48.584566 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:48.584566 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:48.584566 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:48.584566 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:48.584566 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:48.584566 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:48.584566 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:48.584566 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:48.588836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 235/451] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Nov 2 07:15:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:48.589586 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b4e4a-0022-4cd9-9450-4a8078faaabf None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:48.601852 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a75a67-cb28-473e-ab3c-926c815abb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:48.602478 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a75a67-cb28-473e-ab3c-926c815abb85 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:48.602792 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a75a67-cb28-473e-ab3c-926c815abb85 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:48.603062 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a75a67-cb28-473e-ab3c-926c815abb85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:48.631556 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:48.674968 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a75a67-cb28-473e-ab3c-926c815abb85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:48.676583 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a75a67-cb28-473e-ab3c-926c815abb85 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:48.706155 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b4e4a-0022-4cd9-9450-4a8078faaabf None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:48.743345 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93396e4b-a678-4fbf-9d9c-7b70805dbbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:48.743977 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93396e4b-a678-4fbf-9d9c-7b70805dbbe2 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:48.744252 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93396e4b-a678-4fbf-9d9c-7b70805dbbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:48.744526 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93396e4b-a678-4fbf-9d9c-7b70805dbbe2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:48.755440 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93396e4b-a678-4fbf-9d9c-7b70805dbbe2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:48.760416 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-93396e4b-a678-4fbf-9d9c-7b70805dbbe2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:48.760416 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:48.760416 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:48.760416 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:48.760416 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:48.760416 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:48.760416 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:48.760416 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:48.760416 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:48.760416 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:48.760416 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:48.760416 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:48.760416 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:48.760416 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:48.760416 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:48.760416 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:48.761293 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:48.761293 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:48.761293 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:48.761293 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:48.761293 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:48.761293 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:48.761293 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:48.761293 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:48.761293 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:48.761293 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:48.761293 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:48.761293 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:48.761293 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:48.761293 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:48.761293 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:48.761293 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:48.762070 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:48.762070 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:48.762070 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:48.762070 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:48.762070 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:48.762070 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:48.762070 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:48.762070 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:48.762070 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:48.762070 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:48.762070 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:48.762070 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:48.762070 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 236/452] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Nov 2 07:15:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:48.763425 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93396e4b-a678-4fbf-9d9c-7b70805dbbe2 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:48.777801 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:48.793704 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 217/453] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:15:48.793704 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a75a67-cb28-473e-ab3c-926c815abb85 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:48.868010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:48.877587 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93396e4b-a678-4fbf-9d9c-7b70805dbbe2 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:48.898894 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a75a67-cb28-473e-ab3c-926c815abb85 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:48.916510 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3b33f-cefb-4b4e-8fbb-d0ec1898b5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:48.917150 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3b33f-cefb-4b4e-8fbb-d0ec1898b5a4 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:48.917427 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3b33f-cefb-4b4e-8fbb-d0ec1898b5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:48.917696 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3b33f-cefb-4b4e-8fbb-d0ec1898b5a4 None admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:48.918453 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f3b33f-cefb-4b4e-8fbb-d0ec1898b5a4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:48.918792 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f3b33f-cefb-4b4e-8fbb-d0ec1898b5a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:48.920392 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f3b33f-cefb-4b4e-8fbb-d0ec1898b5a4 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:48.934438 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d573f-1916-4a19-a99a-8a8cfe53b911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:48.935394 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d573f-1916-4a19-a99a-8a8cfe53b911 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:48.935675 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d573f-1916-4a19-a99a-8a8cfe53b911 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:48.935946 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d573f-1916-4a19-a99a-8a8cfe53b911 None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:48.936645 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367d573f-1916-4a19-a99a-8a8cfe53b911 None admin] RBAC: Adding query filter params (name=service) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:48.936918 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367d573f-1916-4a19-a99a-8a8cfe53b911 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:48.938256 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367d573f-1916-4a19-a99a-8a8cfe53b911 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:48.943422 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 237/454] 10.222.0.30 () {56 vars in 1137 bytes} [Wed Nov 2 07:15:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:48.943422 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3b33f-cefb-4b4e-8fbb-d0ec1898b5a4 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:48.956838 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 218/455] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Nov 2 07:15:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:48.956838 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d573f-1916-4a19-a99a-8a8cfe53b911 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:49.012763 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:49.047018 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3b33f-cefb-4b4e-8fbb-d0ec1898b5a4 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:49.060486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d573f-1916-4a19-a99a-8a8cfe53b911 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:49.080751 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c054c1a1-ec13-4986-8f46-d7837d7da2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:49.081362 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c054c1a1-ec13-4986-8f46-d7837d7da2fb None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:49.081637 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c054c1a1-ec13-4986-8f46-d7837d7da2fb None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:49.081907 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c054c1a1-ec13-4986-8f46-d7837d7da2fb None admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:49.082559 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c054c1a1-ec13-4986-8f46-d7837d7da2fb None admin] RBAC: Adding query filter params (name=service) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:49.082889 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c054c1a1-ec13-4986-8f46-d7837d7da2fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:49.084300 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c054c1a1-ec13-4986-8f46-d7837d7da2fb None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:49.092475 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 238/456] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Nov 2 07:15:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:49.093267 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c054c1a1-ec13-4986-8f46-d7837d7da2fb None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:49.102559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff2f71-20c5-4d36-aa2f-ca5c4f597905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:49.103435 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff2f71-20c5-4d36-aa2f-ca5c4f597905 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:49.103877 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff2f71-20c5-4d36-aa2f-ca5c4f597905 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:49.104334 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff2f71-20c5-4d36-aa2f-ca5c4f597905 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:49.112365 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:49.117555 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff2f71-20c5-4d36-aa2f-ca5c4f597905 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:49.119041 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff2f71-20c5-4d36-aa2f-ca5c4f597905 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:49.134033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-2dff2f71-20c5-4d36-aa2f-ca5c4f597905 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:49.134033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:49.134033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:49.134033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:49.134033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:49.134033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:49.134033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:49.134033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:49.134033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:49.134033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:49.134033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:49.134033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:49.134033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:49.134033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:49.134033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:49.134033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:49.134972 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:49.134972 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:49.134972 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:49.134972 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:49.134972 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:49.134972 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:49.134972 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:49.134972 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:49.134972 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:49.134972 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:49.134972 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:49.134972 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:49.134972 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:49.134972 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:49.134972 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:49.134972 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:49.134972 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:49.135811 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:49.135811 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:49.135811 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:49.135811 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 219/457] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:49.136436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff2f71-20c5-4d36-aa2f-ca5c4f597905 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:49.203611 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c054c1a1-ec13-4986-8f46-d7837d7da2fb None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:49.239523 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3deb1-305f-45d0-892b-f469433b2216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:49.240049 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3deb1-305f-45d0-892b-f469433b2216 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:49.240380 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3deb1-305f-45d0-892b-f469433b2216 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:49.240472 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff2f71-20c5-4d36-aa2f-ca5c4f597905 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:49.240648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3deb1-305f-45d0-892b-f469433b2216 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:49.247399 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:49.275173 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7905c5bd-58bd-4d19-8c0e-d6a38efbe539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:49.275805 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905c5bd-58bd-4d19-8c0e-d6a38efbe539 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:49.276082 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905c5bd-58bd-4d19-8c0e-d6a38efbe539 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:49.276351 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905c5bd-58bd-4d19-8c0e-d6a38efbe539 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:49.287921 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7905c5bd-58bd-4d19-8c0e-d6a38efbe539 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:49.289370 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7905c5bd-58bd-4d19-8c0e-d6a38efbe539 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:49.305081 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-7905c5bd-58bd-4d19-8c0e-d6a38efbe539 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:49.305081 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:49.305081 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:49.305081 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:49.305081 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:49.305081 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:49.305081 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:49.305081 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:49.305081 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:49.305081 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:49.305081 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:49.305081 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:49.305081 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:49.305081 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:49.305081 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:49.305081 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:49.305977 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:49.305977 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:49.305977 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:49.305977 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:49.305977 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:49.305977 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:49.305977 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:49.305977 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:49.305977 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:49.305977 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:49.305977 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:49.305977 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:49.305977 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:49.305977 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:49.305977 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:49.305977 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:49.305977 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:49.307241 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:49.307241 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:49.307241 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:49.307241 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 220/458] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:49.307241 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7905c5bd-58bd-4d19-8c0e-d6a38efbe539 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:49.317632 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3deb1-305f-45d0-892b-f469433b2216 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:49.319138 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3deb1-305f-45d0-892b-f469433b2216 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:49.333501 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:49.419944 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7905c5bd-58bd-4d19-8c0e-d6a38efbe539 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:49.424334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 239/459] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:15:49.425341 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3deb1-305f-45d0-892b-f469433b2216 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:49.457938 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157452c8-fcf2-42c1-b2bb-a39b734dfed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:49.458574 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157452c8-fcf2-42c1-b2bb-a39b734dfed5 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:49.458923 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157452c8-fcf2-42c1-b2bb-a39b734dfed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:49.459196 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157452c8-fcf2-42c1-b2bb-a39b734dfed5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:49.469846 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157452c8-fcf2-42c1-b2bb-a39b734dfed5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:49.471393 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157452c8-fcf2-42c1-b2bb-a39b734dfed5 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:49.486709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-157452c8-fcf2-42c1-b2bb-a39b734dfed5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:49.486709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:49.486709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:49.486709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:49.486709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:49.486709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:49.486709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:49.486709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:49.486709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:49.486709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:49.486709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:49.486709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:49.486709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:49.486709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:49.486709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:49.486709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:49.487696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:49.487696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:49.487696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:49.487696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:49.487696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:49.487696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:49.487696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:49.487696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:49.487696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:49.487696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:49.487696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:49.487696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:49.487696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:49.487696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:49.487696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:49.487696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:49.487696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:49.488537 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:49.488537 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:49.488537 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:49.488537 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 221/460] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:49.488537 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157452c8-fcf2-42c1-b2bb-a39b734dfed5 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:49.506826 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:49.527059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3deb1-305f-45d0-892b-f469433b2216 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:49.542275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:49.563516 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbbe0a4-66d5-46cf-b2eb-20f5cd3797c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:49.564156 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbbe0a4-66d5-46cf-b2eb-20f5cd3797c5 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:49.564431 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbbe0a4-66d5-46cf-b2eb-20f5cd3797c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:49.564699 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbbe0a4-66d5-46cf-b2eb-20f5cd3797c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:49.565327 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbbe0a4-66d5-46cf-b2eb-20f5cd3797c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:49.565627 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbbe0a4-66d5-46cf-b2eb-20f5cd3797c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:49.567027 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbbe0a4-66d5-46cf-b2eb-20f5cd3797c5 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:49.578756 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 240/461] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:49.579569 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbbe0a4-66d5-46cf-b2eb-20f5cd3797c5 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:49.611503 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157452c8-fcf2-42c1-b2bb-a39b734dfed5 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:49.645489 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42080d2b-df79-4f6b-8bf6-8834aed819fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:49.646094 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42080d2b-df79-4f6b-8bf6-8834aed819fb None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:49.646371 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42080d2b-df79-4f6b-8bf6-8834aed819fb None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:49.646638 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42080d2b-df79-4f6b-8bf6-8834aed819fb None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:49.647297 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42080d2b-df79-4f6b-8bf6-8834aed819fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:49.647596 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42080d2b-df79-4f6b-8bf6-8834aed819fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:49.648966 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42080d2b-df79-4f6b-8bf6-8834aed819fb None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:49.661494 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 222/462] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:49.662317 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42080d2b-df79-4f6b-8bf6-8834aed819fb None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:49.692199 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbbe0a4-66d5-46cf-b2eb-20f5cd3797c5 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:49.726044 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb3e941-49b1-47e7-9856-e5ac5a2cf791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:49.726740 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3e941-49b1-47e7-9856-e5ac5a2cf791 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:49.727015 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3e941-49b1-47e7-9856-e5ac5a2cf791 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:49.727552 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3e941-49b1-47e7-9856-e5ac5a2cf791 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:49.738163 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:49.738870 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb3e941-49b1-47e7-9856-e5ac5a2cf791 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:49.741204 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb3e941-49b1-47e7-9856-e5ac5a2cf791 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:49.758498 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-8fb3e941-49b1-47e7-9856-e5ac5a2cf791 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:49.758498 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:49.758498 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:49.758498 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:49.758498 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:49.758498 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:49.758498 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:49.758498 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:49.758498 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:49.758498 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:49.758498 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:49.758498 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:49.758498 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:49.758498 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:49.758498 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:49.758498 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:49.759568 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:49.759568 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:49.759568 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:49.759568 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:49.759568 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:49.759568 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:49.759568 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:49.759568 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:49.759568 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:49.759568 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:49.759568 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:49.759568 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:49.759568 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:49.759568 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:49.759568 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:49.759568 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:49.759568 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:49.765837 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:49.765837 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:49.765837 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:49.765837 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 241/463] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:49.765837 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb3e941-49b1-47e7-9856-e5ac5a2cf791 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:49.772095 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42080d2b-df79-4f6b-8bf6-8834aed819fb None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:49.793122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:49.811048 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c778ae7e-58df-4f0d-b276-029b1f79e06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:49.811753 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778ae7e-58df-4f0d-b276-029b1f79e06c None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:49.812029 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778ae7e-58df-4f0d-b276-029b1f79e06c None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:49.812301 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778ae7e-58df-4f0d-b276-029b1f79e06c None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:49.812937 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c778ae7e-58df-4f0d-b276-029b1f79e06c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:49.813236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c778ae7e-58df-4f0d-b276-029b1f79e06c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:49.814634 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c778ae7e-58df-4f0d-b276-029b1f79e06c None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:49.826568 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 223/464] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:49.827638 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c778ae7e-58df-4f0d-b276-029b1f79e06c None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:49.876073 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb3e941-49b1-47e7-9856-e5ac5a2cf791 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:49.911904 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77e27c-a3bb-4b91-a9bb-98df8b4708af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:49.912531 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77e27c-a3bb-4b91-a9bb-98df8b4708af None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:49.912806 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77e27c-a3bb-4b91-a9bb-98df8b4708af None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:49.913073 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77e27c-a3bb-4b91-a9bb-98df8b4708af None admin] PATH_INFO: `/v3/projects/service` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:49.920443 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f77e27c-a3bb-4b91-a9bb-98df8b4708af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:49.921924 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f77e27c-a3bb-4b91-a9bb-98df8b4708af None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:49.926018 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c778ae7e-58df-4f0d-b276-029b1f79e06c None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:49.933727 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-0f77e27c-a3bb-4b91-a9bb-98df8b4708af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:49.933727 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:49.933727 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:49.933727 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:49.933727 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:49.933727 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:49.933727 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:49.933727 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:49.933727 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:49.933727 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:49.933727 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:49.933727 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:49.933727 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:15:49.933727 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:15:49.933727 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:15:49.934575 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:15:49.934575 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:49.934575 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:49.934575 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:49.934575 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:49.934575 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:49.934575 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:49.934575 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:49.934575 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:49.934575 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:49.934575 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:49.934575 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:49.934575 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:49.934575 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:49.934575 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:49.934575 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:49.934575 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:49.935465 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:15:49.935465 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:15:49.935465 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:15:49.935465 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:15:49.935465 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:15:49.935465 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:15:49.935465 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:49.935465 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:49.935465 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 242/465] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:49.935934 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77e27c-a3bb-4b91-a9bb-98df8b4708af None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:49.961156 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:49.968517 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d04862-b197-490e-9a88-1ec0bdf3c63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:49.969179 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d04862-b197-490e-9a88-1ec0bdf3c63e None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:49.969457 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d04862-b197-490e-9a88-1ec0bdf3c63e None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:49.969725 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d04862-b197-490e-9a88-1ec0bdf3c63e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:49.978169 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d04862-b197-490e-9a88-1ec0bdf3c63e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:49.979649 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d04862-b197-490e-9a88-1ec0bdf3c63e None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:49.991182 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-15d04862-b197-490e-9a88-1ec0bdf3c63e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 07:15:49.991182 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:49.991182 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:49.991182 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:49.991182 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:49.991182 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:49.991182 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:49.991182 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:49.991182 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:49.991182 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:49.991182 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:49.991182 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:49.991182 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:49.991182 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:49.991182 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:49.991182 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:49.992063 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:49.992063 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:49.992063 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:49.992063 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:49.992063 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:49.992063 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:49.992063 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:49.992063 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:49.992063 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:49.992063 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:49.992063 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:49.992063 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:49.992063 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:49.992063 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:49.992063 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:49.992063 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:49.992063 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:49.992063 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:49.992930 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:49.992930 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:49.992930 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:49.992930 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:49.992930 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:49.992930 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:49.992930 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:49.992930 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:49.992930 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 07:15:49.992930 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:49.992930 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 224/466] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:49.993508 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d04862-b197-490e-9a88-1ec0bdf3c63e None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:50.031212 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:50.047900 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77e27c-a3bb-4b91-a9bb-98df8b4708af None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:50.084034 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868efc39-1a5f-4869-b1a5-8590b357dca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:50.084652 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868efc39-1a5f-4869-b1a5-8590b357dca4 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:50.084928 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868efc39-1a5f-4869-b1a5-8590b357dca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:50.085198 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868efc39-1a5f-4869-b1a5-8590b357dca4 None admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:50.085830 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868efc39-1a5f-4869-b1a5-8590b357dca4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:50.086135 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868efc39-1a5f-4869-b1a5-8590b357dca4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:50.087556 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868efc39-1a5f-4869-b1a5-8590b357dca4 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:50.099573 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 243/467] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:50.100214 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868efc39-1a5f-4869-b1a5-8590b357dca4 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:50.112800 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d04862-b197-490e-9a88-1ec0bdf3c63e None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:50.147480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87ad33-a752-40e8-8a5e-6392d486e46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:50.148140 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87ad33-a752-40e8-8a5e-6392d486e46d None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:50.148418 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87ad33-a752-40e8-8a5e-6392d486e46d None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:50.148693 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87ad33-a752-40e8-8a5e-6392d486e46d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:50.157634 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a87ad33-a752-40e8-8a5e-6392d486e46d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:50.159153 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a87ad33-a752-40e8-8a5e-6392d486e46d None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:50.170221 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-1a87ad33-a752-40e8-8a5e-6392d486e46d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Nov 02 07:15:50.170221 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:50.170221 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:50.170221 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:50.170221 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:50.170221 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:50.170221 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:50.170221 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:50.170221 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:50.170221 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:50.170221 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:50.170221 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:50.170221 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:50.170221 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:50.170221 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:50.170221 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:50.171327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:50.171327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:50.171327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:50.171327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:50.171327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:50.171327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:50.171327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:50.171327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:50.171327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:50.171327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:50.171327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:50.171327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:50.171327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:50.171327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:50.171327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:50.171327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:50.171327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:50.171327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:50.172328 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:50.172328 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:50.172328 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:50.172328 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:50.172328 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:50.172328 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:50.172328 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:50.172328 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:50.172328 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Nov 02 07:15:50.172328 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:50.174800 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 225/468] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Nov 2 07:15:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:50.175775 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87ad33-a752-40e8-8a5e-6392d486e46d None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:50.203193 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868efc39-1a5f-4869-b1a5-8590b357dca4 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:50.226211 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:50.237644 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942c5a16-1480-4b3c-b91a-8b143fecc8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:50.238258 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c5a16-1480-4b3c-b91a-8b143fecc8e2 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:50.238538 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c5a16-1480-4b3c-b91a-8b143fecc8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:50.238841 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c5a16-1480-4b3c-b91a-8b143fecc8e2 None admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:50.239526 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c5a16-1480-4b3c-b91a-8b143fecc8e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:50.239854 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c5a16-1480-4b3c-b91a-8b143fecc8e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:50.241319 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c5a16-1480-4b3c-b91a-8b143fecc8e2 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:50.252633 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 244/469] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 07:15:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:50.257599 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942c5a16-1480-4b3c-b91a-8b143fecc8e2 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:50.262045 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:50.281585 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87ad33-a752-40e8-8a5e-6392d486e46d None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:50.315461 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d7973-1c99-4377-841a-9c42091e8379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:50.316066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d7973-1c99-4377-841a-9c42091e8379 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:50.316342 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d7973-1c99-4377-841a-9c42091e8379 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:50.316605 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d7973-1c99-4377-841a-9c42091e8379 None admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:50.317315 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d7973-1c99-4377-841a-9c42091e8379 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:50.317612 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d7973-1c99-4377-841a-9c42091e8379 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:50.319051 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d7973-1c99-4377-841a-9c42091e8379 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:50.336322 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 226/470] 10.222.0.30 () {56 vars in 1137 bytes} [Wed Nov 2 07:15:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:50.337702 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f5a8c-9582-4954-ac7f-14647c8b7220 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:50.337973 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f5a8c-9582-4954-ac7f-14647c8b7220 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:50.338264 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f5a8c-9582-4954-ac7f-14647c8b7220 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:50.342707 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 227/471] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:50.342707 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f5a8c-9582-4954-ac7f-14647c8b7220 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:50.358577 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942c5a16-1480-4b3c-b91a-8b143fecc8e2 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:50.393147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f82191-9566-43b2-9423-f42dbefa911d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:50.393789 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f82191-9566-43b2-9423-f42dbefa911d None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:50.394059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f82191-9566-43b2-9423-f42dbefa911d None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:50.394312 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f82191-9566-43b2-9423-f42dbefa911d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:50.400783 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f82191-9566-43b2-9423-f42dbefa911d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:50.402219 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f82191-9566-43b2-9423-f42dbefa911d None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:50.413651 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-69f82191-9566-43b2-9423-f42dbefa911d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:50.413651 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:50.413651 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:50.413651 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:50.413651 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:50.413651 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:50.413651 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:50.413651 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:50.413651 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:50.413651 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:50.413651 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:50.413651 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:50.413651 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:15:50.413651 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:15:50.413651 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:15:50.414706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:15:50.414706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:50.414706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:50.414706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:50.414706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:50.414706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:50.414706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:50.414706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:50.414706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:50.414706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:50.414706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:50.414706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:50.414706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:50.414706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:50.414706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:50.414706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:50.414706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:50.415698 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:15:50.415698 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:15:50.415698 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:15:50.415698 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:15:50.415698 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:15:50.415698 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:15:50.415698 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:50.415698 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:50.415698 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 245/472] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:50.415698 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f82191-9566-43b2-9423-f42dbefa911d None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:50.442167 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f5a8c-9582-4954-ac7f-14647c8b7220 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:50.451477 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:50.476051 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59936e5-9501-477f-8b21-254daaed7dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:50.476669 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59936e5-9501-477f-8b21-254daaed7dde None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:50.476942 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59936e5-9501-477f-8b21-254daaed7dde None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:50.477206 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59936e5-9501-477f-8b21-254daaed7dde None admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:50.477919 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59936e5-9501-477f-8b21-254daaed7dde None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:50.478219 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59936e5-9501-477f-8b21-254daaed7dde None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:50.479681 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59936e5-9501-477f-8b21-254daaed7dde None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:50.480602 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:50.496653 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 228/473] 10.222.0.30 () {56 vars in 1139 bytes} [Wed Nov 2 07:15:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:50.497541 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59936e5-9501-477f-8b21-254daaed7dde None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:50.520755 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f82191-9566-43b2-9423-f42dbefa911d None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:50.556833 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63848de6-f9aa-4ee3-8dbd-55cb8adbf265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:50.557490 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63848de6-f9aa-4ee3-8dbd-55cb8adbf265 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:50.557863 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63848de6-f9aa-4ee3-8dbd-55cb8adbf265 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:50.558254 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63848de6-f9aa-4ee3-8dbd-55cb8adbf265 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:50.559106 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63848de6-f9aa-4ee3-8dbd-55cb8adbf265 None admin] RBAC: Adding query filter params (role.id=268fecadd98446f2b7d41755f3fe3649, scope.project.id=25e2bc8417754f909b416b93d5b7afb7, user.id=e6495385dccc4fa48c05b1feb957846c) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:50.559500 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63848de6-f9aa-4ee3-8dbd-55cb8adbf265 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:50.561130 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63848de6-f9aa-4ee3-8dbd-55cb8adbf265 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:50.567488 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 246/474] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 07:15:50 2022] GET /identity/v3/role_assignments?user.id=e6495385dccc4fa48c05b1feb957846c&scope.project.id=25e2bc8417754f909b416b93d5b7afb7&role.id=268fecadd98446f2b7d41755f3fe3649 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:50.568629 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652a718-3c45-4b1e-8b6d-8bff811a991c None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:50.568933 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652a718-3c45-4b1e-8b6d-8bff811a991c None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:50.569203 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652a718-3c45-4b1e-8b6d-8bff811a991c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:50.590800 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-f652a718-3c45-4b1e-8b6d-8bff811a991c None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:50.592241 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-f652a718-3c45-4b1e-8b6d-8bff811a991c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:50.608220 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59936e5-9501-477f-8b21-254daaed7dde None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:50.651014 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae1164-3835-4b85-aee0-e9877fa0868a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:50.651892 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae1164-3835-4b85-aee0-e9877fa0868a None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:50.652356 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae1164-3835-4b85-aee0-e9877fa0868a None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:50.652789 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae1164-3835-4b85-aee0-e9877fa0868a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:50.668105 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 247/475] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:50.668989 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652a718-3c45-4b1e-8b6d-8bff811a991c None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:50.720662 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:50.746021 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:50.760007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcae1164-3835-4b85-aee0-e9877fa0868a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:50.761787 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcae1164-3835-4b85-aee0-e9877fa0868a None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:50.797321 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652a718-3c45-4b1e-8b6d-8bff811a991c None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:50.844049 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c44b3d-0328-41a8-9b52-106d65a2de17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:50.844587 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c44b3d-0328-41a8-9b52-106d65a2de17 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:50.844865 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c44b3d-0328-41a8-9b52-106d65a2de17 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:50.845134 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c44b3d-0328-41a8-9b52-106d65a2de17 None admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:50.845831 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c44b3d-0328-41a8-9b52-106d65a2de17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:50.846128 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c44b3d-0328-41a8-9b52-106d65a2de17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:50.847611 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c44b3d-0328-41a8-9b52-106d65a2de17 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:50.860059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 248/476] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 07:15:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:50.860954 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c44b3d-0328-41a8-9b52-106d65a2de17 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:50.881670 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 229/477] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:15:50.882373 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae1164-3835-4b85-aee0-e9877fa0868a None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:50.959431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:50.985426 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c44b3d-0328-41a8-9b52-106d65a2de17 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:50.993540 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae1164-3835-4b85-aee0-e9877fa0868a None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:51.008235 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:51.026363 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2652d-3031-426c-880b-5d22a9e2ef28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:51.027027 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2652d-3031-426c-880b-5d22a9e2ef28 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:51.027305 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2652d-3031-426c-880b-5d22a9e2ef28 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:51.027577 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2652d-3031-426c-880b-5d22a9e2ef28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:51.033365 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a7f81-ad8f-407b-81e5-c3a93e6c26bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:51.034199 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a7f81-ad8f-407b-81e5-c3a93e6c26bc None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:51.034687 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a7f81-ad8f-407b-81e5-c3a93e6c26bc None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:51.035165 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a7f81-ad8f-407b-81e5-c3a93e6c26bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:51.036087 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a7f81-ad8f-407b-81e5-c3a93e6c26bc None admin] RBAC: Adding query filter params (role.id=268fecadd98446f2b7d41755f3fe3649, scope.project.id=25e2bc8417754f909b416b93d5b7afb7, user.id=9a051ca8154540ad93db6d028b8323a0) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:51.036582 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a7f81-ad8f-407b-81e5-c3a93e6c26bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:51.038219 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a7f81-ad8f-407b-81e5-c3a93e6c26bc None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:51.053442 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 230/478] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 07:15:50 2022] GET /identity/v3/role_assignments?user.id=9a051ca8154540ad93db6d028b8323a0&scope.project.id=25e2bc8417754f909b416b93d5b7afb7&role.id=268fecadd98446f2b7d41755f3fe3649 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:51.054320 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a7f81-ad8f-407b-81e5-c3a93e6c26bc None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:51.118626 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f2652d-3031-426c-880b-5d22a9e2ef28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:51.120538 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f2652d-3031-426c-880b-5d22a9e2ef28 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:51.178769 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a7f81-ad8f-407b-81e5-c3a93e6c26bc None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:51.209770 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:51.217616 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f165d6-2e3b-4064-904c-72b5d7ea9f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:51.218954 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f165d6-2e3b-4064-904c-72b5d7ea9f2d None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:51.219433 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f165d6-2e3b-4064-904c-72b5d7ea9f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:51.219893 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f165d6-2e3b-4064-904c-72b5d7ea9f2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:51.233806 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f165d6-2e3b-4064-904c-72b5d7ea9f2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:51.236069 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f165d6-2e3b-4064-904c-72b5d7ea9f2d None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:51.247711 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 249/479] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:15:51.248586 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2652d-3031-426c-880b-5d22a9e2ef28 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:51.257319 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-59f165d6-2e3b-4064-904c-72b5d7ea9f2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:51.257319 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:51.257319 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:51.257319 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:51.257319 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:51.257319 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:51.257319 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:51.257319 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:51.257319 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:51.257319 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:51.257319 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:51.257319 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:51.257319 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:51.257319 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:51.257319 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:51.257319 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:51.258212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:51.258212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:51.258212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:51.258212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:51.258212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:51.258212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:51.258212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:51.258212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:51.258212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:51.258212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:51.258212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:51.258212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:51.258212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:51.258212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:51.258212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:51.258212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:51.258212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:51.259468 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:51.259468 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:51.259468 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:51.259468 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 231/480] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:51.259692 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f165d6-2e3b-4064-904c-72b5d7ea9f2d None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:51.272409 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:51.378194 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2652d-3031-426c-880b-5d22a9e2ef28 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:51.380697 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f165d6-2e3b-4064-904c-72b5d7ea9f2d None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:51.416388 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3ffff1-a87d-4b0a-b2bd-edf6042b1c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:51.417020 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ffff1-a87d-4b0a-b2bd-edf6042b1c8d None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:51.417284 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ffff1-a87d-4b0a-b2bd-edf6042b1c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:51.417547 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ffff1-a87d-4b0a-b2bd-edf6042b1c8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:51.423318 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57caf111-3fc2-4b84-ad12-b9ac74e3019a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:51.423962 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57caf111-3fc2-4b84-ad12-b9ac74e3019a None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:51.424237 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57caf111-3fc2-4b84-ad12-b9ac74e3019a None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:51.424506 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57caf111-3fc2-4b84-ad12-b9ac74e3019a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:51.429290 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3ffff1-a87d-4b0a-b2bd-edf6042b1c8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:51.430873 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3ffff1-a87d-4b0a-b2bd-edf6042b1c8d None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:51.437865 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57caf111-3fc2-4b84-ad12-b9ac74e3019a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:51.443610 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-57caf111-3fc2-4b84-ad12-b9ac74e3019a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:51.443610 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:51.443610 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:51.443610 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:51.443610 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:51.443610 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:51.443610 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:51.443610 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:51.443610 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:51.443610 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:51.443610 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:51.443610 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:51.443610 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:51.443610 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:51.443610 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:51.443610 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:51.444604 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:51.444604 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:51.444604 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:51.444604 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:51.444604 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:51.444604 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:51.444604 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:51.444604 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:51.444604 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:51.444604 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:51.444604 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:51.444604 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:51.444604 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:51.444604 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:51.444604 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:51.444604 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:51.445792 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:51.445792 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:51.445792 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:51.445792 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:51.445792 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:51.445792 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:51.445792 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:51.445792 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:51.445792 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:51.445792 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:51.445792 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:51.445792 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:51.445792 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 250/481] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Nov 2 07:15:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:51.446436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-2e3ffff1-a87d-4b0a-b2bd-edf6042b1c8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:51.446436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:51.446436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:51.446436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:51.446436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:51.446436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:51.446436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:51.446436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:51.446436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:51.446436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:51.446436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:51.446436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:51.446436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:51.446436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:51.446436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:51.446436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:51.447848 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:51.447848 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:51.447848 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:51.447848 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:51.447848 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:51.447848 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:51.447848 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:51.447848 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:51.447848 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:51.447848 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:51.447848 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:51.447848 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:51.447848 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:51.447848 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:51.447848 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:51.447848 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:51.447848 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:51.448865 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:51.448865 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:51.448865 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:51.449069 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57caf111-3fc2-4b84-ad12-b9ac74e3019a None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:51.449148 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 232/482] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:51.452056 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3ffff1-a87d-4b0a-b2bd-edf6042b1c8d None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:51.491364 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:51.551317 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:51.572107 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57caf111-3fc2-4b84-ad12-b9ac74e3019a None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:51.583377 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3ffff1-a87d-4b0a-b2bd-edf6042b1c8d None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:51.614643 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d3442b-d78d-423a-addd-9235e3a0756f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:51.615297 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3442b-d78d-423a-addd-9235e3a0756f None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:51.615567 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3442b-d78d-423a-addd-9235e3a0756f None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:51.615829 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3442b-d78d-423a-addd-9235e3a0756f None admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:51.616467 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d3442b-d78d-423a-addd-9235e3a0756f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:51.616758 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d3442b-d78d-423a-addd-9235e3a0756f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:51.618129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d3442b-d78d-423a-addd-9235e3a0756f None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:51.630822 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 251/483] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:51.631394 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d3442b-d78d-423a-addd-9235e3a0756f None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:51.637729 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e5810-74d9-495c-aad4-266eec29e316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:51.638354 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e5810-74d9-495c-aad4-266eec29e316 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:51.638621 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e5810-74d9-495c-aad4-266eec29e316 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:51.640517 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e5810-74d9-495c-aad4-266eec29e316 None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:51.642830 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e5810-74d9-495c-aad4-266eec29e316 None admin] RBAC: Adding query filter params (name=service) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:51.643406 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e5810-74d9-495c-aad4-266eec29e316 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:51.645006 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e5810-74d9-495c-aad4-266eec29e316 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:51.656890 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 233/484] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Nov 2 07:15:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:51.657742 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e5810-74d9-495c-aad4-266eec29e316 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:51.763052 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d3442b-d78d-423a-addd-9235e3a0756f None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:51.764940 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:51.771275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e5810-74d9-495c-aad4-266eec29e316 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:51.809018 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a268e-3b0a-402e-ac7f-b0f303f884ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:51.809898 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a268e-3b0a-402e-ac7f-b0f303f884ab None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:51.810271 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a268e-3b0a-402e-ac7f-b0f303f884ab None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:51.810757 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a268e-3b0a-402e-ac7f-b0f303f884ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:51.811721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08daf3-8b94-4709-b04f-42fc3c4687c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:51.812542 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08daf3-8b94-4709-b04f-42fc3c4687c3 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:51.812995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08daf3-8b94-4709-b04f-42fc3c4687c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:51.813656 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08daf3-8b94-4709-b04f-42fc3c4687c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:51.814862 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08daf3-8b94-4709-b04f-42fc3c4687c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:51.815300 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08daf3-8b94-4709-b04f-42fc3c4687c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:51.816818 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08daf3-8b94-4709-b04f-42fc3c4687c3 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:51.818870 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a268e-3b0a-402e-ac7f-b0f303f884ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:51.820548 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a268e-3b0a-402e-ac7f-b0f303f884ab None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:51.831562 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 252/485] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:51.832703 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08daf3-8b94-4709-b04f-42fc3c4687c3 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:51.833840 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-722a268e-3b0a-402e-ac7f-b0f303f884ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:51.833840 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:51.833840 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:51.833840 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:51.833840 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:51.833840 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:51.833840 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:51.833840 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:51.833840 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:51.833840 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:51.833840 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:51.833840 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:51.833840 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:15:51.833840 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:15:51.833840 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:15:51.834996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:15:51.834996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:51.834996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:51.834996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:51.834996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:51.834996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:51.834996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:51.834996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:51.834996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:51.834996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:51.834996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:51.834996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:51.834996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:51.834996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:51.834996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:51.834996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:51.834996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:51.836124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:15:51.836124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:15:51.836124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:15:51.836124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:15:51.836124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:15:51.836124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:15:51.836124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:51.836124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:51.842272 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 234/486] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:51.842272 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a268e-3b0a-402e-ac7f-b0f303f884ab None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:51.842945 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:51.954902 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08daf3-8b94-4709-b04f-42fc3c4687c3 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:51.975121 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a268e-3b0a-402e-ac7f-b0f303f884ab None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:51.995543 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721a70da-5f6e-4154-99ab-d89a26165b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:51.997172 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a70da-5f6e-4154-99ab-d89a26165b53 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:51.997617 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a70da-5f6e-4154-99ab-d89a26165b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:51.998024 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a70da-5f6e-4154-99ab-d89a26165b53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:52.013481 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721a70da-5f6e-4154-99ab-d89a26165b53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:52.014113 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5f0fa-62ad-4b28-8743-f23d86d10b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:52.014892 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5f0fa-62ad-4b28-8743-f23d86d10b44 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:52.015298 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5f0fa-62ad-4b28-8743-f23d86d10b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:52.015651 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721a70da-5f6e-4154-99ab-d89a26165b53 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:52.015760 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5f0fa-62ad-4b28-8743-f23d86d10b44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:52.022947 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5f0fa-62ad-4b28-8743-f23d86d10b44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:52.024560 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5f0fa-62ad-4b28-8743-f23d86d10b44 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:52.033255 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-721a70da-5f6e-4154-99ab-d89a26165b53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:52.033255 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:52.033255 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:52.033255 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:52.033255 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:52.033255 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:52.033255 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:52.033255 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:52.033255 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:52.033255 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:52.033255 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:52.033255 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:52.033255 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:52.033255 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:52.033255 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:52.033255 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:52.034337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:52.034337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:52.034337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:52.034337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:52.034337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:52.034337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:52.034337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:52.034337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:52.034337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:52.034337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:52.034337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:52.034337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:52.034337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:52.034337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:52.034337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:52.034337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:52.034337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:52.035417 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:52.035417 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:52.035417 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:52.037130 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 253/487] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:52.038051 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721a70da-5f6e-4154-99ab-d89a26165b53 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:52.044090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-a3e5f0fa-62ad-4b28-8743-f23d86d10b44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:52.044090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:52.044090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:52.044090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:52.044090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:52.044090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:52.044090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:52.044090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:52.044090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:52.044090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:52.044090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:52.044090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:52.044090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:15:52.044090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:15:52.044090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:15:52.045049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:15:52.045049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:52.045049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:52.045049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:52.045049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:52.045049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:52.045049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:52.045049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:52.045049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:52.045049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:52.045049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:52.045049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:52.045049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:52.045049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:52.045049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:52.045049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:52.045049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:52.046059 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:15:52.046059 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:15:52.046059 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:15:52.046059 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:15:52.046059 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:15:52.046059 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:15:52.046059 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:52.046059 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:52.047377 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 235/488] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:52.048314 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5f0fa-62ad-4b28-8743-f23d86d10b44 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:52.070099 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:52.107749 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:52.155643 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5f0fa-62ad-4b28-8743-f23d86d10b44 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:52.160322 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721a70da-5f6e-4154-99ab-d89a26165b53 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:52.189889 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe50e48-b57b-4f3a-83cc-3dfb7f84ddbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:52.190533 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe50e48-b57b-4f3a-83cc-3dfb7f84ddbd None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:52.190907 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe50e48-b57b-4f3a-83cc-3dfb7f84ddbd None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:52.191247 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe50e48-b57b-4f3a-83cc-3dfb7f84ddbd None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:52.192091 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe50e48-b57b-4f3a-83cc-3dfb7f84ddbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:52.192336 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe50e48-b57b-4f3a-83cc-3dfb7f84ddbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:52.193834 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe50e48-b57b-4f3a-83cc-3dfb7f84ddbd None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:52.198191 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a44eb2-abe7-4882-b4ba-5b61c0ab218e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:52.199027 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a44eb2-abe7-4882-b4ba-5b61c0ab218e None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:52.199492 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a44eb2-abe7-4882-b4ba-5b61c0ab218e None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:52.200026 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a44eb2-abe7-4882-b4ba-5b61c0ab218e None admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:52.200976 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a44eb2-abe7-4882-b4ba-5b61c0ab218e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:52.201547 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a44eb2-abe7-4882-b4ba-5b61c0ab218e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:52.203180 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a44eb2-abe7-4882-b4ba-5b61c0ab218e None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:52.211123 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 236/489] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:52.212343 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe50e48-b57b-4f3a-83cc-3dfb7f84ddbd None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:52.217817 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 254/490] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 07:15:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:52.219002 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a44eb2-abe7-4882-b4ba-5b61c0ab218e None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:52.292785 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:52.317357 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe50e48-b57b-4f3a-83cc-3dfb7f84ddbd None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:52.329251 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a44eb2-abe7-4882-b4ba-5b61c0ab218e None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:52.343908 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:52.353732 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe24a9-7627-4423-8449-eabfaabd3937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:52.354525 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe24a9-7627-4423-8449-eabfaabd3937 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:52.354901 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe24a9-7627-4423-8449-eabfaabd3937 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:52.355260 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe24a9-7627-4423-8449-eabfaabd3937 None admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:52.356034 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfe24a9-7627-4423-8449-eabfaabd3937 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:52.356626 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfe24a9-7627-4423-8449-eabfaabd3937 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:52.358122 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfe24a9-7627-4423-8449-eabfaabd3937 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:52.370337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f0f8b-247e-42d7-8fb0-e1c780b9abfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:52.371292 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f0f8b-247e-42d7-8fb0-e1c780b9abfc None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:52.371502 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 237/491] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 07:15:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:52.372221 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f0f8b-247e-42d7-8fb0-e1c780b9abfc None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:52.372656 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe24a9-7627-4423-8449-eabfaabd3937 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:52.372856 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f0f8b-247e-42d7-8fb0-e1c780b9abfc None admin] PATH_INFO: `/v3/users/swift` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:52.381144 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f0f8b-247e-42d7-8fb0-e1c780b9abfc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:52.382862 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f0f8b-247e-42d7-8fb0-e1c780b9abfc None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:52.395645 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-471f0f8b-247e-42d7-8fb0-e1c780b9abfc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Nov 02 07:15:52.395645 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:52.395645 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:52.395645 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:52.395645 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:52.395645 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:52.395645 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:52.395645 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:52.395645 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:52.395645 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:52.395645 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:52.395645 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:52.395645 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:52.395645 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:52.395645 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:52.395645 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:52.396771 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:52.396771 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:52.396771 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:52.396771 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:52.396771 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:52.396771 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:52.396771 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:52.396771 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:52.396771 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:52.396771 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:52.396771 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:52.396771 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:52.396771 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:52.396771 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:52.396771 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:52.396771 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:52.396771 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:52.396771 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:52.397660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:52.397660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:52.397660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:52.397660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:52.397660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:52.397660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:52.397660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:52.397660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:52.397660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Nov 02 07:15:52.397660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:52.399962 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 255/492] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 07:15:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:52.401011 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f0f8b-247e-42d7-8fb0-e1c780b9abfc None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:52.489521 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe24a9-7627-4423-8449-eabfaabd3937 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:52.507963 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f0f8b-247e-42d7-8fb0-e1c780b9abfc None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:52.525950 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f6244-f291-45e3-bd47-97bafa1444c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:52.526713 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f6244-f291-45e3-bd47-97bafa1444c7 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:52.527000 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f6244-f291-45e3-bd47-97bafa1444c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:52.527328 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f6244-f291-45e3-bd47-97bafa1444c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:52.528324 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f6244-f291-45e3-bd47-97bafa1444c7 None admin] RBAC: Adding query filter params (role.id=268fecadd98446f2b7d41755f3fe3649, scope.project.id=25e2bc8417754f909b416b93d5b7afb7, user.id=315861510b1041dca19d1e8753d9b21d) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:52.528525 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f6244-f291-45e3-bd47-97bafa1444c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:52.529982 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f6244-f291-45e3-bd47-97bafa1444c7 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:52.533305 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:52.537299 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 238/493] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 07:15:52 2022] GET /identity/v3/role_assignments?user.id=315861510b1041dca19d1e8753d9b21d&scope.project.id=25e2bc8417754f909b416b93d5b7afb7&role.id=268fecadd98446f2b7d41755f3fe3649 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:52.538200 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f6244-f291-45e3-bd47-97bafa1444c7 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:52.549938 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c572d-f8f3-48fa-9f96-782434d1daf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:52.551078 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c572d-f8f3-48fa-9f96-782434d1daf5 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:52.551550 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c572d-f8f3-48fa-9f96-782434d1daf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:52.552017 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c572d-f8f3-48fa-9f96-782434d1daf5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:52.552988 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9c572d-f8f3-48fa-9f96-782434d1daf5 None admin] RBAC: Adding query filter params (role.id=268fecadd98446f2b7d41755f3fe3649, scope.project.id=25e2bc8417754f909b416b93d5b7afb7, user.id=65d51ec74e1e4c25bcc4365eb858bac0) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:52.553486 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9c572d-f8f3-48fa-9f96-782434d1daf5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:52.555351 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9c572d-f8f3-48fa-9f96-782434d1daf5 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:52.562815 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 256/494] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 07:15:52 2022] GET /identity/v3/role_assignments?user.id=65d51ec74e1e4c25bcc4365eb858bac0&scope.project.id=25e2bc8417754f909b416b93d5b7afb7&role.id=268fecadd98446f2b7d41755f3fe3649 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:52.675982 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f6244-f291-45e3-bd47-97bafa1444c7 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:52.717305 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87086b07-c130-4373-be7b-fbaa28ec7757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:52.717935 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87086b07-c130-4373-be7b-fbaa28ec7757 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:52.718214 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87086b07-c130-4373-be7b-fbaa28ec7757 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:52.718483 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87086b07-c130-4373-be7b-fbaa28ec7757 None admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:52.719335 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87086b07-c130-4373-be7b-fbaa28ec7757 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:52.719642 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87086b07-c130-4373-be7b-fbaa28ec7757 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:52.721111 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87086b07-c130-4373-be7b-fbaa28ec7757 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:52.742127 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 239/495] 10.222.0.30 () {56 vars in 1135 bytes} [Wed Nov 2 07:15:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:52.748395 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c572d-f8f3-48fa-9f96-782434d1daf5 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:52.795356 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:52.872880 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c572d-f8f3-48fa-9f96-782434d1daf5 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:52.909374 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2bad8e-1e73-4026-ad89-bca41db78684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:52.910001 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2bad8e-1e73-4026-ad89-bca41db78684 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:52.910275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2bad8e-1e73-4026-ad89-bca41db78684 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:52.910547 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2bad8e-1e73-4026-ad89-bca41db78684 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:52.935752 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118543d-d9b3-46e7-b680-b08318b206cd None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:52.936408 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118543d-d9b3-46e7-b680-b08318b206cd None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:52.936907 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118543d-d9b3-46e7-b680-b08318b206cd None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:52.940155 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 240/496] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:52.944957 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd439812-337d-497e-891a-f5e552d0d7a5 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:52.945313 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd439812-337d-497e-891a-f5e552d0d7a5 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:52.945653 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd439812-337d-497e-891a-f5e552d0d7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:52.970508 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-dd439812-337d-497e-891a-f5e552d0d7a5 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:52.971854 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-dd439812-337d-497e-891a-f5e552d0d7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:53.018219 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2bad8e-1e73-4026-ad89-bca41db78684 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:53.020424 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2bad8e-1e73-4026-ad89-bca41db78684 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:53.076058 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:53.078546 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 241/497] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:15:53.099790 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:53.171944 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 257/498] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 07:15:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 07:15:53.177681 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd439812-337d-497e-891a-f5e552d0d7a5 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:53.315017 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd439812-337d-497e-891a-f5e552d0d7a5 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:53.356614 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efa58ac-30a9-4ecb-baf1-16438e505760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:53.357249 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa58ac-30a9-4ecb-baf1-16438e505760 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:53.357521 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa58ac-30a9-4ecb-baf1-16438e505760 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:53.357785 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa58ac-30a9-4ecb-baf1-16438e505760 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:53.374606 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efa58ac-30a9-4ecb-baf1-16438e505760 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:53.376094 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efa58ac-30a9-4ecb-baf1-16438e505760 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:53.391295 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-6efa58ac-30a9-4ecb-baf1-16438e505760 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:53.391295 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:53.391295 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:53.391295 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:53.391295 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:53.391295 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:53.391295 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:53.391295 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:53.391295 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:53.391295 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:53.391295 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:53.391295 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:53.391295 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:53.391295 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:53.391295 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:53.391295 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:53.392236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:53.392236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:53.392236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:53.392236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:53.392236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:53.392236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:53.392236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:53.392236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:53.392236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:53.392236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:53.392236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:53.392236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:53.392236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:53.392236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:53.392236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:53.392236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:53.392236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:53.393085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:53.393085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:53.393085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:53.393085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 242/499] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:53.398201 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2bad8e-1e73-4026-ad89-bca41db78684 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:53.436355 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f7de8-3b70-4173-800a-b9e85a1e86ab None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:53.436709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f7de8-3b70-4173-800a-b9e85a1e86ab None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:53.436899 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f7de8-3b70-4173-800a-b9e85a1e86ab None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:53.438370 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 243/500] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:15:53.443679 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff817b9-0ffd-479c-8021-e975069f9ba0 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:53.443957 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff817b9-0ffd-479c-8021-e975069f9ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:53.444225 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff817b9-0ffd-479c-8021-e975069f9ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:53.474694 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-8ff817b9-0ffd-479c-8021-e975069f9ba0 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:53.475960 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-8ff817b9-0ffd-479c-8021-e975069f9ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:53.519381 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:53.519844 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2bad8e-1e73-4026-ad89-bca41db78684 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:53.553808 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 244/501] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:15:53.554715 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff817b9-0ffd-479c-8021-e975069f9ba0 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:53.555017 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf8c00-6cfc-4c65-a2aa-4a86738a5bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:53.555870 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf8c00-6cfc-4c65-a2aa-4a86738a5bfa None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:53.556356 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf8c00-6cfc-4c65-a2aa-4a86738a5bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:53.556818 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf8c00-6cfc-4c65-a2aa-4a86738a5bfa None admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:53.557648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badf8c00-6cfc-4c65-a2aa-4a86738a5bfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:53.558140 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badf8c00-6cfc-4c65-a2aa-4a86738a5bfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:53.560116 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badf8c00-6cfc-4c65-a2aa-4a86738a5bfa None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:53.571106 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:53.573754 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 258/502] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:53.579811 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf8c00-6cfc-4c65-a2aa-4a86738a5bfa None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:53.725128 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff817b9-0ffd-479c-8021-e975069f9ba0 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:53.726224 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf8c00-6cfc-4c65-a2aa-4a86738a5bfa None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:53.772329 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b02fc-9609-4791-b2bf-e0b1ab3fd98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:53.773370 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b02fc-9609-4791-b2bf-e0b1ab3fd98e None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:53.773884 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b02fc-9609-4791-b2bf-e0b1ab3fd98e None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:53.774374 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b02fc-9609-4791-b2bf-e0b1ab3fd98e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:53.789444 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b02fc-9609-4791-b2bf-e0b1ab3fd98e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:53.791234 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b02fc-9609-4791-b2bf-e0b1ab3fd98e None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:53.796119 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506319c2-6ccd-4c50-a7a2-84afe0f773bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:53.796983 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506319c2-6ccd-4c50-a7a2-84afe0f773bc None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:53.797441 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506319c2-6ccd-4c50-a7a2-84afe0f773bc None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:53.797889 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506319c2-6ccd-4c50-a7a2-84afe0f773bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:53.817947 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506319c2-6ccd-4c50-a7a2-84afe0f773bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:53.818617 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-b69b02fc-9609-4791-b2bf-e0b1ab3fd98e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:53.818617 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:53.818617 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:53.818617 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:53.818617 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:53.818617 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:53.818617 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:53.818617 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:53.818617 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:53.818617 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:53.818617 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:53.818617 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:53.818617 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:15:53.818617 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:15:53.818617 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:15:53.819967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:15:53.819967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:53.819967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:53.819967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:53.819967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:53.819967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:53.819967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:53.819967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:53.819967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:53.819967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:53.819967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:53.819967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:53.819967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:53.819967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:53.819967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:53.819967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:53.819967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:53.820827 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:15:53.820827 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:15:53.820827 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:15:53.820827 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:15:53.820827 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:15:53.820827 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:15:53.820827 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:53.820827 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:53.824333 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 259/503] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:53.827112 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-506319c2-6ccd-4c50-a7a2-84afe0f773bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:53.827112 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:53.827112 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:53.827112 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:53.827112 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:53.827112 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:53.827112 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:53.827112 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:53.827112 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:53.827112 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:53.827112 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:53.827112 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:53.827112 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:53.827112 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:53.827112 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:53.827112 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:53.828006 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:53.828006 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:53.828006 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:53.828006 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:53.828006 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:53.828006 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:53.828006 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:53.828006 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:53.828006 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:53.828006 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:53.828006 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:53.828006 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:53.828006 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:53.828006 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:53.828006 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:53.828006 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:53.828819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:53.828819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:53.828819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:53.828819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:53.828819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:53.828819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:53.828819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:53.828819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:53.828819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:53.828819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:53.828819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:53.828819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:53.829964 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b02fc-9609-4791-b2bf-e0b1ab3fd98e None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:53.831584 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 245/504] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 07:15:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:53.839567 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506319c2-6ccd-4c50-a7a2-84afe0f773bc None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:53.894126 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:53.924476 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:53.973805 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506319c2-6ccd-4c50-a7a2-84afe0f773bc None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:53.985491 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b02fc-9609-4791-b2bf-e0b1ab3fd98e None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:54.008542 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f913ae-b472-45b6-a3c1-3e368063e7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:54.009239 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f913ae-b472-45b6-a3c1-3e368063e7a3 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:54.009583 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f913ae-b472-45b6-a3c1-3e368063e7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:54.009913 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f913ae-b472-45b6-a3c1-3e368063e7a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:54.010639 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f913ae-b472-45b6-a3c1-3e368063e7a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:54.011026 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f913ae-b472-45b6-a3c1-3e368063e7a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:54.012584 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f913ae-b472-45b6-a3c1-3e368063e7a3 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:54.025609 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 246/505] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:54.025609 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f913ae-b472-45b6-a3c1-3e368063e7a3 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:54.033144 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765d329-d925-4ffe-95e7-f19c0a270b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:54.033855 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765d329-d925-4ffe-95e7-f19c0a270b60 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:54.034237 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765d329-d925-4ffe-95e7-f19c0a270b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:54.034576 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765d329-d925-4ffe-95e7-f19c0a270b60 None admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:54.035353 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765d329-d925-4ffe-95e7-f19c0a270b60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:54.035768 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765d329-d925-4ffe-95e7-f19c0a270b60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:54.037264 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765d329-d925-4ffe-95e7-f19c0a270b60 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:54.050137 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 260/506] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 07:15:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:54.051120 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765d329-d925-4ffe-95e7-f19c0a270b60 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:54.135336 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:54.150036 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f913ae-b472-45b6-a3c1-3e368063e7a3 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:54.166309 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765d329-d925-4ffe-95e7-f19c0a270b60 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:54.189448 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:54.195530 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22880127-786d-4452-8bf9-1c28fed0a381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:54.196388 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22880127-786d-4452-8bf9-1c28fed0a381 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:54.197060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22880127-786d-4452-8bf9-1c28fed0a381 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:54.198021 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22880127-786d-4452-8bf9-1c28fed0a381 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:54.213516 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22880127-786d-4452-8bf9-1c28fed0a381 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:54.215007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22880127-786d-4452-8bf9-1c28fed0a381 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:54.215773 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e00e3-b51a-4400-a417-b3fa37a7a2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:54.217001 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e00e3-b51a-4400-a417-b3fa37a7a2c5 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:54.217850 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e00e3-b51a-4400-a417-b3fa37a7a2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:54.218368 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e00e3-b51a-4400-a417-b3fa37a7a2c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:54.230942 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-22880127-786d-4452-8bf9-1c28fed0a381 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:54.230942 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:54.230942 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:54.230942 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:54.230942 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:54.230942 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:54.230942 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:54.230942 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:54.230942 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:54.230942 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:54.230942 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:54.230942 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:54.230942 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:54.230942 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:54.230942 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:54.230942 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:54.231935 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:54.231935 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:54.231935 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:54.231935 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:54.231935 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:54.231935 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:54.231935 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:54.231935 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:54.231935 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:54.231935 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:54.231935 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:54.231935 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:54.231935 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:54.231935 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:54.231935 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:54.231935 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:54.231935 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:54.232831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:54.232831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:54.232831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:54.233851 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e00e3-b51a-4400-a417-b3fa37a7a2c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:54.235600 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e00e3-b51a-4400-a417-b3fa37a7a2c5 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:54.237349 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 247/507] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:54.238326 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22880127-786d-4452-8bf9-1c28fed0a381 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:54.257275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-ab5e00e3-b51a-4400-a417-b3fa37a7a2c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:54.257275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:54.257275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:54.257275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:54.257275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:54.257275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:54.257275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:54.257275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:54.257275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:54.257275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:54.257275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:54.257275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:54.257275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:54.257275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:54.257275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:54.257275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:54.258269 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:54.258269 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:54.258269 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:54.258269 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:54.258269 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:54.258269 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:54.258269 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:54.258269 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:54.258269 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:54.258269 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:54.258269 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:54.258269 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:54.258269 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:54.258269 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:54.258269 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:54.258269 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:54.258269 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:54.259235 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:54.259235 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:54.259235 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:54.264602 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 261/508] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:54.264602 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e00e3-b51a-4400-a417-b3fa37a7a2c5 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:54.352683 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22880127-786d-4452-8bf9-1c28fed0a381 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:54.387512 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01dbb8-181e-4622-b12a-f3fc739e41fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:54.387801 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01dbb8-181e-4622-b12a-f3fc739e41fc None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:54.388018 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01dbb8-181e-4622-b12a-f3fc739e41fc None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:54.388288 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01dbb8-181e-4622-b12a-f3fc739e41fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:54.389007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee01dbb8-181e-4622-b12a-f3fc739e41fc None admin] RBAC: Adding query filter params (role.id=268fecadd98446f2b7d41755f3fe3649, scope.project.id=25e2bc8417754f909b416b93d5b7afb7, user.id=836c4044aaf74de391219289b57d2956) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:54.389312 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee01dbb8-181e-4622-b12a-f3fc739e41fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:54.390845 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee01dbb8-181e-4622-b12a-f3fc739e41fc None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:54.402143 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e00e3-b51a-4400-a417-b3fa37a7a2c5 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:54.402989 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 248/509] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 07:15:54 2022] GET /identity/v3/role_assignments?user.id=836c4044aaf74de391219289b57d2956&scope.project.id=25e2bc8417754f909b416b93d5b7afb7&role.id=268fecadd98446f2b7d41755f3fe3649 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:54.403993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01dbb8-181e-4622-b12a-f3fc739e41fc None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:54.424806 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:54.447471 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cce0d1-9b0e-48a9-9f1f-373eee6b8db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:54.448490 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cce0d1-9b0e-48a9-9f1f-373eee6b8db8 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:54.448997 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cce0d1-9b0e-48a9-9f1f-373eee6b8db8 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:54.449513 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cce0d1-9b0e-48a9-9f1f-373eee6b8db8 None admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:54.450525 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cce0d1-9b0e-48a9-9f1f-373eee6b8db8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:54.451067 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cce0d1-9b0e-48a9-9f1f-373eee6b8db8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:54.453213 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cce0d1-9b0e-48a9-9f1f-373eee6b8db8 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:54.469231 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 262/510] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:54.478948 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cce0d1-9b0e-48a9-9f1f-373eee6b8db8 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:54.540345 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:54.631363 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01dbb8-181e-4622-b12a-f3fc739e41fc None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:54.675769 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fcae6-4189-4c3a-8e64-c69367663430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:54.676362 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fcae6-4189-4c3a-8e64-c69367663430 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:54.676641 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fcae6-4189-4c3a-8e64-c69367663430 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:54.676910 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fcae6-4189-4c3a-8e64-c69367663430 None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:54.677551 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23fcae6-4189-4c3a-8e64-c69367663430 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:54.677845 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23fcae6-4189-4c3a-8e64-c69367663430 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:54.679254 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23fcae6-4189-4c3a-8e64-c69367663430 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:54.686987 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cce0d1-9b0e-48a9-9f1f-373eee6b8db8 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:54.693367 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 249/511] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:54.700183 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fcae6-4189-4c3a-8e64-c69367663430 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:54.729900 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1dd97-681f-4a98-b0e1-634d2c6c7dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:54.730632 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1dd97-681f-4a98-b0e1-634d2c6c7dda None admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:54.731053 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1dd97-681f-4a98-b0e1-634d2c6c7dda None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:54.731326 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1dd97-681f-4a98-b0e1-634d2c6c7dda None admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:54.732091 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca1dd97-681f-4a98-b0e1-634d2c6c7dda None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:54.733662 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca1dd97-681f-4a98-b0e1-634d2c6c7dda None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:54.760341 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 263/512] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 07:15:54 2022] POST /identity/v3/users => generated 254 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:15:54.779305 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:54.818581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fcae6-4189-4c3a-8e64-c69367663430 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:54.864505 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a1f45-0c8a-413c-884f-b11f88e1ca45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:54.865152 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a1f45-0c8a-413c-884f-b11f88e1ca45 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:54.865422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a1f45-0c8a-413c-884f-b11f88e1ca45 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:54.865684 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a1f45-0c8a-413c-884f-b11f88e1ca45 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:54.874601 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a1f45-0c8a-413c-884f-b11f88e1ca45 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:54.876131 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a1f45-0c8a-413c-884f-b11f88e1ca45 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:54.895181 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-118a1f45-0c8a-413c-884f-b11f88e1ca45 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:15:54.895181 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:54.895181 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:54.895181 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:54.895181 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:54.895181 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:54.895181 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:54.895181 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:54.895181 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:54.895181 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:54.895181 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:54.895181 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:54.895181 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:54.895181 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:54.895181 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:54.895181 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:54.896233 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:54.896233 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:54.896233 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:54.896233 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:54.896233 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:54.896233 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:54.896233 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:54.896233 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:54.896233 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:54.896233 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:54.896233 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:54.896233 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:54.896233 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:54.896233 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:54.896233 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:54.896233 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:54.896233 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:54.896233 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:54.897143 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:54.897143 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:54.897143 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:54.897143 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:54.897143 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:54.897143 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:54.897143 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:54.897143 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:54.897143 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:15:54.897143 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:54.897143 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 250/513] 10.222.0.30 () {56 vars in 1093 bytes} [Wed Nov 2 07:15:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:54.902168 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1dd97-681f-4a98-b0e1-634d2c6c7dda None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:55.028423 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:55.046384 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1dd97-681f-4a98-b0e1-634d2c6c7dda None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:55.082853 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f960c7-3cf1-4c4d-9079-6d57f2c552ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:55.083484 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f960c7-3cf1-4c4d-9079-6d57f2c552ac None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:55.083754 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f960c7-3cf1-4c4d-9079-6d57f2c552ac None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:55.084023 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f960c7-3cf1-4c4d-9079-6d57f2c552ac None admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:55.084754 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f960c7-3cf1-4c4d-9079-6d57f2c552ac None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:55.085055 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f960c7-3cf1-4c4d-9079-6d57f2c552ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:55.086501 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f960c7-3cf1-4c4d-9079-6d57f2c552ac None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:55.110077 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 264/514] 10.222.0.30 () {56 vars in 1133 bytes} [Wed Nov 2 07:15:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:55.111547 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a1f45-0c8a-413c-884f-b11f88e1ca45 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:55.142396 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55e8e1-871e-4387-8bb4-87cc9c211be4 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:55.142700 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55e8e1-871e-4387-8bb4-87cc9c211be4 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:55.143073 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55e8e1-871e-4387-8bb4-87cc9c211be4 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:55.144808 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 265/515] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:15:55.151558 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e312ac9-dc95-4ff0-9e53-f454b69b31b6 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:55.151949 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e312ac9-dc95-4ff0-9e53-f454b69b31b6 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:55.152228 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e312ac9-dc95-4ff0-9e53-f454b69b31b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:55.176569 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-9e312ac9-dc95-4ff0-9e53-f454b69b31b6 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:55.177966 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-9e312ac9-dc95-4ff0-9e53-f454b69b31b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:55.241628 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a1f45-0c8a-413c-884f-b11f88e1ca45 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:55.271130 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 266/516] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:15:55.275086 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:55.281055 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac09c3-7623-4ac8-a4ae-fb661d67ca48 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:55.281285 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac09c3-7623-4ac8-a4ae-fb661d67ca48 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:55.281518 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac09c3-7623-4ac8-a4ae-fb661d67ca48 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:55.282701 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e683b68-4fcf-4191-9e43-3e507d7f4f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:55.283258 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 267/517] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:15:55.283428 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e683b68-4fcf-4191-9e43-3e507d7f4f31 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:55.283665 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e683b68-4fcf-4191-9e43-3e507d7f4f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:55.283909 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e683b68-4fcf-4191-9e43-3e507d7f4f31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:55.295387 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e683b68-4fcf-4191-9e43-3e507d7f4f31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:55.296380 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff6ef2-e485-434e-927a-5b0323171a09 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:55.296655 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff6ef2-e485-434e-927a-5b0323171a09 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:55.296870 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e683b68-4fcf-4191-9e43-3e507d7f4f31 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:55.297028 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff6ef2-e485-434e-927a-5b0323171a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:55.312198 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-3e683b68-4fcf-4191-9e43-3e507d7f4f31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:55.312198 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:55.312198 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:55.312198 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:55.312198 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:55.312198 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:55.312198 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:55.312198 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:55.312198 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:55.312198 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:55.312198 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:55.312198 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:55.312198 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:55.312198 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:55.312198 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:55.312198 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:55.314337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:55.314337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:55.314337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:55.314337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:55.314337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:55.314337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:55.314337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:55.314337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:55.314337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:55.314337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:55.314337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:55.314337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:55.314337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:55.314337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:55.314337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:55.314337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:55.314337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:55.316406 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:55.316406 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:55.316406 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:55.318695 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 251/518] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:55.319514 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e683b68-4fcf-4191-9e43-3e507d7f4f31 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:55.355794 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-4cff6ef2-e485-434e-927a-5b0323171a09 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:55.357096 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-4cff6ef2-e485-434e-927a-5b0323171a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:55.425277 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e683b68-4fcf-4191-9e43-3e507d7f4f31 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:55.460891 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1afd5b-4019-4ec8-ac0b-60ae80104084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:55.461503 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1afd5b-4019-4ec8-ac0b-60ae80104084 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:55.461778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1afd5b-4019-4ec8-ac0b-60ae80104084 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:55.462052 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1afd5b-4019-4ec8-ac0b-60ae80104084 None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:55.462731 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1afd5b-4019-4ec8-ac0b-60ae80104084 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:55.463034 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1afd5b-4019-4ec8-ac0b-60ae80104084 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:55.464737 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1afd5b-4019-4ec8-ac0b-60ae80104084 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:55.482173 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 252/519] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:55.483514 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1afd5b-4019-4ec8-ac0b-60ae80104084 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:55.527035 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:55.564595 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 268/520] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 07:15:55.565840 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff6ef2-e485-434e-927a-5b0323171a09 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:55.568350 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:55.589840 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1afd5b-4019-4ec8-ac0b-60ae80104084 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:55.637339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060873ca-e416-4926-8d85-37f65206887c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:55.638027 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060873ca-e416-4926-8d85-37f65206887c None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:55.638309 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060873ca-e416-4926-8d85-37f65206887c None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:55.638578 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060873ca-e416-4926-8d85-37f65206887c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:55.645951 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060873ca-e416-4926-8d85-37f65206887c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:55.647583 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060873ca-e416-4926-8d85-37f65206887c None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:55.659779 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-060873ca-e416-4926-8d85-37f65206887c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:55.659779 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:55.659779 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:55.659779 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:55.659779 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:55.659779 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:55.659779 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:55.659779 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:55.659779 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:55.659779 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:55.659779 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:55.659779 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:55.659779 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:15:55.659779 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:15:55.659779 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:15:55.660717 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:15:55.660717 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:55.660717 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:55.660717 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:55.660717 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:55.660717 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:55.660717 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:55.660717 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:55.660717 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:55.660717 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:55.660717 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:55.660717 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:55.660717 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:55.660717 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:55.660717 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:55.660717 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:55.660717 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:55.661597 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:15:55.661597 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:15:55.661597 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:15:55.661597 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:15:55.661597 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:15:55.661597 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:15:55.661597 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:55.661597 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:55.665701 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 253/521] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:55.666986 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060873ca-e416-4926-8d85-37f65206887c None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:55.794725 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:55.799846 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060873ca-e416-4926-8d85-37f65206887c None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:55.812266 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff6ef2-e485-434e-927a-5b0323171a09 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:55.858854 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16148ba6-91cb-4ee2-8d79-5d5f518462e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:55.859569 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16148ba6-91cb-4ee2-8d79-5d5f518462e2 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:55.859742 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c369116-2726-485a-a5a7-2dde8561d998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:55.859884 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16148ba6-91cb-4ee2-8d79-5d5f518462e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:55.860117 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16148ba6-91cb-4ee2-8d79-5d5f518462e2 None admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:55.860534 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c369116-2726-485a-a5a7-2dde8561d998 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:55.860806 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16148ba6-91cb-4ee2-8d79-5d5f518462e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:55.860914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c369116-2726-485a-a5a7-2dde8561d998 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:55.861110 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16148ba6-91cb-4ee2-8d79-5d5f518462e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:55.861266 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c369116-2726-485a-a5a7-2dde8561d998 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:55.861904 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c369116-2726-485a-a5a7-2dde8561d998 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:55.862574 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16148ba6-91cb-4ee2-8d79-5d5f518462e2 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:55.863395 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c369116-2726-485a-a5a7-2dde8561d998 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:55.874337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-6c369116-2726-485a-a5a7-2dde8561d998 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:15:55.874337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:55.874337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:15:55.874337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:15:55.874337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:15:55.874337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:15:55.874337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:15:55.874337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:55.874337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:15:55.874337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:55.874337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:15:55.874337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:55.874337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:55.874337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:55.874337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:55.874337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:55.874337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:55.875371 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:55.875371 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:55.875371 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:55.875371 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:55.875371 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:55.875371 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:55.875371 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:15:55.875371 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:15:55.875371 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:15:55.875371 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:15:55.875371 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:55.875371 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:55.875371 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:55.875371 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:55.875371 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:55.875371 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:55.876808 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:55.876808 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:55.876808 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:55.876808 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:55.876808 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:55.876808 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:55.876808 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:55.876808 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:55.876808 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:55.876808 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:55.876808 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:15:55.876808 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:55.876808 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:15:55.876808 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:55.876808 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 269/522] 10.222.0.30 () {56 vars in 1129 bytes} [Wed Nov 2 07:15:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:55.877861 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c369116-2726-485a-a5a7-2dde8561d998 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:55.882348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 254/523] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 07:15:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:55.883263 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16148ba6-91cb-4ee2-8d79-5d5f518462e2 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:55.945584 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:55.985031 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c369116-2726-485a-a5a7-2dde8561d998 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:55.997287 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16148ba6-91cb-4ee2-8d79-5d5f518462e2 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:56.023446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b1ff31-5df5-488e-8175-831c76f5ede6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:56.024134 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1ff31-5df5-488e-8175-831c76f5ede6 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:56.024409 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1ff31-5df5-488e-8175-831c76f5ede6 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:56.024676 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1ff31-5df5-488e-8175-831c76f5ede6 None admin] PATH_INFO: `/v3/services/network` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:56.025291 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b1ff31-5df5-488e-8175-831c76f5ede6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:56.026682 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b1ff31-5df5-488e-8175-831c76f5ede6 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:56.036009 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-d2b1ff31-5df5-488e-8175-831c76f5ede6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:15:56.036009 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:56.036009 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:15:56.036009 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:15:56.036009 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:15:56.036009 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:15:56.036009 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:15:56.036009 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:56.036009 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:15:56.036009 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:56.036009 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:15:56.036009 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:56.036009 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:56.036009 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:56.036009 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:56.036009 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:56.036009 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:56.038800 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:56.038800 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:56.038800 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:56.038800 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:56.038800 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:56.038800 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:56.038800 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:15:56.038800 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:15:56.038800 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:15:56.038800 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:15:56.038800 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:56.038800 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:56.038800 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:56.038800 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:56.038800 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:56.038800 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:56.040828 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:56.040828 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:56.040828 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:56.040828 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:56.040828 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:56.040828 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:56.040828 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:56.040828 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:56.040828 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:56.040828 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:56.040828 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:15:56.040828 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:15:56.040828 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:15:56.040828 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:56.041743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e0ba5-a970-4205-8e52-ce31fb74a74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:56.041857 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 270/524] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:55 2022] GET /identity/v3/services/network => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:56.041939 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e0ba5-a970-4205-8e52-ce31fb74a74b None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:56.041939 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e0ba5-a970-4205-8e52-ce31fb74a74b None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:56.042084 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b1ff31-5df5-488e-8175-831c76f5ede6 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:56.042163 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e0ba5-a970-4205-8e52-ce31fb74a74b None admin] PATH_INFO: `/v3/services` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:56.050328 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42e0ba5-a970-4205-8e52-ce31fb74a74b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:56.050328 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42e0ba5-a970-4205-8e52-ce31fb74a74b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:56.050328 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42e0ba5-a970-4205-8e52-ce31fb74a74b None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:56.059742 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 255/525] 10.222.0.30 () {56 vars in 1124 bytes} [Wed Nov 2 07:15:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:56.061083 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e0ba5-a970-4205-8e52-ce31fb74a74b None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:56.090872 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:56.155983 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b1ff31-5df5-488e-8175-831c76f5ede6 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:56.177262 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e0ba5-a970-4205-8e52-ce31fb74a74b None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:56.186538 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:56.194199 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4fe01c-f2af-4688-ad30-7278bfd09f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:56.194886 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fe01c-f2af-4688-ad30-7278bfd09f89 None admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:56.195159 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fe01c-f2af-4688-ad30-7278bfd09f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:56.195427 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fe01c-f2af-4688-ad30-7278bfd09f89 None admin] PATH_INFO: `/v3/projects/25e2bc8417754f909b416b93d5b7afb7/users/e6495385dccc4fa48c05b1feb957846c/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:56.197404 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4fe01c-f2af-4688-ad30-7278bfd09f89 None admin] RBAC: Authorizing `identity:create_grant(project_id=25e2bc8417754f909b416b93d5b7afb7, user_id=e6495385dccc4fa48c05b1feb957846c, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:56.198981 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4fe01c-f2af-4688-ad30-7278bfd09f89 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:56.214341 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 271/526] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 07:15:56 2022] PUT /identity/v3/projects/25e2bc8417754f909b416b93d5b7afb7/users/e6495385dccc4fa48c05b1feb957846c/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:15:56.215129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4fe01c-f2af-4688-ad30-7278bfd09f89 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:56.218224 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0800f9e8-d21d-4b96-a7a6-d5123cb0d4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:56.219092 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800f9e8-d21d-4b96-a7a6-d5123cb0d4cd None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:56.219364 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800f9e8-d21d-4b96-a7a6-d5123cb0d4cd None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:56.219672 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800f9e8-d21d-4b96-a7a6-d5123cb0d4cd None admin] PATH_INFO: `/v3/services` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:56.221600 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0800f9e8-d21d-4b96-a7a6-d5123cb0d4cd None admin] RBAC: Adding query filter params (name=network) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:56.221916 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0800f9e8-d21d-4b96-a7a6-d5123cb0d4cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:56.223369 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0800f9e8-d21d-4b96-a7a6-d5123cb0d4cd None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:56.230741 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 256/527] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Nov 2 07:15:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:56.237777 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0800f9e8-d21d-4b96-a7a6-d5123cb0d4cd None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:56.394433 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:56.411279 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0800f9e8-d21d-4b96-a7a6-d5123cb0d4cd None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:56.443481 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4fe01c-f2af-4688-ad30-7278bfd09f89 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:56.453257 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49e4b1-193f-4d49-b7f4-8b032f0d97fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:56.454184 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49e4b1-193f-4d49-b7f4-8b032f0d97fd None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:56.454647 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49e4b1-193f-4d49-b7f4-8b032f0d97fd None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:56.455167 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49e4b1-193f-4d49-b7f4-8b032f0d97fd None admin] PATH_INFO: `/v3/services` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:56.455980 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f49e4b1-193f-4d49-b7f4-8b032f0d97fd None admin] RBAC: Adding query filter params (type=network) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:56.456483 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f49e4b1-193f-4d49-b7f4-8b032f0d97fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:56.458578 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f49e4b1-193f-4d49-b7f4-8b032f0d97fd None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:56.468751 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 257/528] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Nov 2 07:15:56 2022] GET /identity/v3/services?type=network => generated 124 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:56.500144 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5115a31-c4ae-4019-80ad-ba41f74f5be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:56.500781 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5115a31-c4ae-4019-80ad-ba41f74f5be9 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:56.501058 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5115a31-c4ae-4019-80ad-ba41f74f5be9 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:56.501326 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5115a31-c4ae-4019-80ad-ba41f74f5be9 None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:56.501972 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5115a31-c4ae-4019-80ad-ba41f74f5be9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:56.502276 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5115a31-c4ae-4019-80ad-ba41f74f5be9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:56.503724 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5115a31-c4ae-4019-80ad-ba41f74f5be9 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:56.510973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 272/529] 10.222.0.30 () {56 vars in 1124 bytes} [Wed Nov 2 07:15:56 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:57.047750 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a20de-3b9f-4c33-a511-05e8fb25dbf1 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:57.048167 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a20de-3b9f-4c33-a511-05e8fb25dbf1 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:57.048340 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a20de-3b9f-4c33-a511-05e8fb25dbf1 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:57.049920 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 258/530] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:57.055025 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4244a8-c415-4489-96cf-62ea026997b7 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:57.055315 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4244a8-c415-4489-96cf-62ea026997b7 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:57.055589 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4244a8-c415-4489-96cf-62ea026997b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:57.077026 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-cc4244a8-c415-4489-96cf-62ea026997b7 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:57.078213 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-cc4244a8-c415-4489-96cf-62ea026997b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:57.147449 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:57.173337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 273/531] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:57.558575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a20de-3b9f-4c33-a511-05e8fb25dbf1 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:57.672242 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a20de-3b9f-4c33-a511-05e8fb25dbf1 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:57.711892 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd2b59-b411-422f-98e7-9b678e109eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:57.712532 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd2b59-b411-422f-98e7-9b678e109eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:57.712810 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd2b59-b411-422f-98e7-9b678e109eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:57.713083 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd2b59-b411-422f-98e7-9b678e109eb0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:57.724026 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd2b59-b411-422f-98e7-9b678e109eb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:57.729234 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-76fd2b59-b411-422f-98e7-9b678e109eb0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:57.729234 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:57.729234 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:57.729234 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:57.729234 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:57.729234 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:57.729234 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:57.729234 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:57.729234 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:57.729234 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:57.729234 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:57.729234 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:57.729234 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:57.729234 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:57.729234 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:57.729234 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:57.730201 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:57.730201 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:57.730201 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:57.730201 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:57.730201 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:57.730201 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:57.730201 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:57.730201 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:57.730201 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:57.730201 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:57.730201 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:57.730201 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:57.730201 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:57.730201 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:57.730201 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:57.730201 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:57.731077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:57.731077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:57.731077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:57.731077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:57.731077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:57.731077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:57.731077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:57.731077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:57.731077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:57.731077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:57.731077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:57.731077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:57.733469 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 259/532] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 07:15:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:57.735220 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba489c6-5204-4ae6-af20-8751c92332b2 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:57.735509 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba489c6-5204-4ae6-af20-8751c92332b2 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:57.735783 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba489c6-5204-4ae6-af20-8751c92332b2 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:57.737352 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 274/533] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:57.737531 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd2b59-b411-422f-98e7-9b678e109eb0 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:57.742296 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97adb3-9f5f-448e-910a-6e88504b5974 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:57.742582 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97adb3-9f5f-448e-910a-6e88504b5974 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:57.742886 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97adb3-9f5f-448e-910a-6e88504b5974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:57.763221 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:57.765511 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-4e97adb3-9f5f-448e-910a-6e88504b5974 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:57.766969 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-4e97adb3-9f5f-448e-910a-6e88504b5974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:57.901681 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 275/534] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:57.912463 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd2b59-b411-422f-98e7-9b678e109eb0 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:57.958203 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e30ed-4e4d-4829-b0db-aabec7f8d8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:57.958850 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e30ed-4e4d-4829-b0db-aabec7f8d8c2 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:57.959127 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e30ed-4e4d-4829-b0db-aabec7f8d8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:57.959396 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e30ed-4e4d-4829-b0db-aabec7f8d8c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:57.960066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42e30ed-4e4d-4829-b0db-aabec7f8d8c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:57.960364 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42e30ed-4e4d-4829-b0db-aabec7f8d8c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:57.961752 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42e30ed-4e4d-4829-b0db-aabec7f8d8c2 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:57.980706 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 260/535] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:15:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:57.986214 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97adb3-9f5f-448e-910a-6e88504b5974 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:58.108849 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97adb3-9f5f-448e-910a-6e88504b5974 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:58.128239 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:58.153006 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5282e6-f7aa-42fe-bd8f-f0ce7adb6806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:58.153990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5282e6-f7aa-42fe-bd8f-f0ce7adb6806 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:58.154845 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5282e6-f7aa-42fe-bd8f-f0ce7adb6806 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:58.155346 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5282e6-f7aa-42fe-bd8f-f0ce7adb6806 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:58.171782 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5282e6-f7aa-42fe-bd8f-f0ce7adb6806 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:58.173547 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5282e6-f7aa-42fe-bd8f-f0ce7adb6806 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:58.194490 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-ba5282e6-f7aa-42fe-bd8f-f0ce7adb6806 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:58.194490 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:58.194490 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:58.194490 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:58.194490 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:58.194490 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:58.194490 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:58.194490 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:58.194490 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:58.194490 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:58.194490 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:58.194490 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:58.194490 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:58.194490 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:58.194490 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:58.194490 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:58.195495 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:58.195495 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:58.195495 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:58.195495 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:58.195495 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:58.195495 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:58.195495 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:58.195495 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:58.195495 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:58.195495 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:58.195495 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:58.195495 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:58.195495 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:58.195495 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:58.195495 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:58.195495 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:58.195495 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:58.196393 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:58.196393 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:58.196393 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:58.198438 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 276/536] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:58.204680 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e30ed-4e4d-4829-b0db-aabec7f8d8c2 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:58.253920 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5282e6-f7aa-42fe-bd8f-f0ce7adb6806 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:58.360670 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:58.381846 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5282e6-f7aa-42fe-bd8f-f0ce7adb6806 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:58.408294 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e30ed-4e4d-4829-b0db-aabec7f8d8c2 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:58.457624 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a292d-0553-4164-94fc-163a233d48ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:58.458270 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a292d-0553-4164-94fc-163a233d48ff None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:58.458545 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a292d-0553-4164-94fc-163a233d48ff None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:58.458845 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a292d-0553-4164-94fc-163a233d48ff None admin] PATH_INFO: `/v3/roles/service` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:58.479750 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7a292d-0553-4164-94fc-163a233d48ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:58.484336 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae72a24d-93b6-4592-b663-624e6298360a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:58.485074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-3e7a292d-0553-4164-94fc-163a233d48ff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:58.485074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:58.485074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:58.485074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:58.485074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:58.485074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:58.485074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:58.485074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:58.485074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:58.485074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:58.485074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:58.485074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:58.485074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:58.485074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:58.485074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:58.485074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:58.485996 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:58.485996 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:58.485996 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:58.485996 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:58.485996 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:58.485996 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:58.485996 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:58.485996 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:58.485996 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:58.485996 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:58.485996 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:58.485996 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:58.485996 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:58.485996 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:58.485996 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:58.485996 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:58.486926 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:58.486926 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:58.486926 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:58.486926 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:58.486926 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:58.486926 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:58.486926 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:58.486926 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:58.486926 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:58.486926 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:58.486926 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:15:58.486926 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:58.486926 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 277/537] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Nov 2 07:15:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:58.487785 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72a24d-93b6-4592-b663-624e6298360a None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:58.488194 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72a24d-93b6-4592-b663-624e6298360a None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:58.488581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72a24d-93b6-4592-b663-624e6298360a None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:58.489349 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae72a24d-93b6-4592-b663-624e6298360a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:58.489768 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae72a24d-93b6-4592-b663-624e6298360a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:58.491301 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae72a24d-93b6-4592-b663-624e6298360a None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:58.498745 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a292d-0553-4164-94fc-163a233d48ff None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:58.509575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 261/538] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:58.514814 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae72a24d-93b6-4592-b663-624e6298360a None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:58.553007 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:58.668450 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae72a24d-93b6-4592-b663-624e6298360a None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:58.684770 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a292d-0553-4164-94fc-163a233d48ff None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:58.703330 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88215ccf-ff91-450e-b154-78fe96c205d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:58.704025 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88215ccf-ff91-450e-b154-78fe96c205d6 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:58.704301 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88215ccf-ff91-450e-b154-78fe96c205d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:58.704569 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88215ccf-ff91-450e-b154-78fe96c205d6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:58.712661 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88215ccf-ff91-450e-b154-78fe96c205d6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:58.714129 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88215ccf-ff91-450e-b154-78fe96c205d6 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:58.715204 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:58.728532 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-88215ccf-ff91-450e-b154-78fe96c205d6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Nov 02 07:15:58.728532 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:58.728532 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:58.728532 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:58.728532 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:58.728532 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:58.728532 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:58.728532 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:58.728532 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:58.728532 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:58.728532 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:58.728532 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:58.728532 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:58.728532 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:58.728532 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:58.728532 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:58.729495 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:58.729495 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:58.729495 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:58.729495 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:58.729495 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:58.729495 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:58.729495 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:58.729495 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:58.729495 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:58.729495 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:58.729495 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:58.729495 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:58.729495 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:58.729495 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:58.729495 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:58.729495 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:58.729495 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:58.729495 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:58.730461 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:58.730461 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:58.730461 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:58.730461 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:58.730461 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:58.730461 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:58.730461 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:58.730461 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:58.730461 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Nov 02 07:15:58.730461 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:58.730461 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 262/539] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 07:15:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:15:58.737460 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88215ccf-ff91-450e-b154-78fe96c205d6 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:58.776321 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98437685-1b44-40f2-8c3f-e2e2ea974657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:58.777070 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98437685-1b44-40f2-8c3f-e2e2ea974657 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:58.777429 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98437685-1b44-40f2-8c3f-e2e2ea974657 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:58.777699 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98437685-1b44-40f2-8c3f-e2e2ea974657 None admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:58.778507 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98437685-1b44-40f2-8c3f-e2e2ea974657 None admin] RBAC: Adding query filter params (name=service) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:58.778835 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98437685-1b44-40f2-8c3f-e2e2ea974657 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:58.780624 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98437685-1b44-40f2-8c3f-e2e2ea974657 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:58.804848 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 278/540] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Nov 2 07:15:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:15:58.812475 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98437685-1b44-40f2-8c3f-e2e2ea974657 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:58.930158 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88215ccf-ff91-450e-b154-78fe96c205d6 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:58.954630 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:58.970413 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e29ea-3b81-44d5-867d-c7ae2de4a67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:58.970778 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98437685-1b44-40f2-8c3f-e2e2ea974657 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:58.971052 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e29ea-3b81-44d5-867d-c7ae2de4a67b None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:58.971326 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e29ea-3b81-44d5-867d-c7ae2de4a67b None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:58.971590 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e29ea-3b81-44d5-867d-c7ae2de4a67b None admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:58.972492 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e29ea-3b81-44d5-867d-c7ae2de4a67b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:58.972795 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e29ea-3b81-44d5-867d-c7ae2de4a67b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:58.975131 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e29ea-3b81-44d5-867d-c7ae2de4a67b None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:58.996351 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 263/541] 10.222.0.30 () {56 vars in 1135 bytes} [Wed Nov 2 07:15:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:58.997700 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c108cc-281b-4673-a870-679fe042d553 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:58.997983 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c108cc-281b-4673-a870-679fe042d553 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:58.998252 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c108cc-281b-4673-a870-679fe042d553 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:59.004500 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 264/542] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:59.006010 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c108cc-281b-4673-a870-679fe042d553 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:59.020269 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9124f0-60d9-495d-a7ee-bf95d37e19ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:59.020937 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9124f0-60d9-495d-a7ee-bf95d37e19ef None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:59.021553 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9124f0-60d9-495d-a7ee-bf95d37e19ef None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:59.021830 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9124f0-60d9-495d-a7ee-bf95d37e19ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:59.038829 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9124f0-60d9-495d-a7ee-bf95d37e19ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:59.041018 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9124f0-60d9-495d-a7ee-bf95d37e19ef None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:59.061273 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-ad9124f0-60d9-495d-a7ee-bf95d37e19ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:59.061273 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:59.061273 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:59.061273 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:59.061273 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:59.061273 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:59.061273 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:59.061273 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:59.061273 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:59.061273 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:59.061273 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:59.061273 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:59.061273 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:59.061273 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:59.061273 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:59.061273 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:59.062298 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:59.062298 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:59.062298 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:59.062298 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:59.062298 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:59.062298 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:59.062298 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:59.062298 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:59.062298 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:59.062298 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:59.062298 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:59.062298 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:59.062298 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:59.062298 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:59.062298 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:59.062298 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:59.062298 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:59.063631 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:59.063631 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:59.063631 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:59.066226 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 279/543] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:59.067685 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eeca46-da6a-4c2b-809c-a4c599a6e5a2 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:59.067964 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eeca46-da6a-4c2b-809c-a4c599a6e5a2 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:59.068312 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eeca46-da6a-4c2b-809c-a4c599a6e5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:59.081134 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:59.094215 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-32eeca46-da6a-4c2b-809c-a4c599a6e5a2 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:59.095780 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-32eeca46-da6a-4c2b-809c-a4c599a6e5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:59.136327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c108cc-281b-4673-a870-679fe042d553 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:59.172913 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 280/544] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:59.173383 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d966f-ebf8-42c7-838a-f15e847b03db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:59.173825 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eeca46-da6a-4c2b-809c-a4c599a6e5a2 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:59.174281 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d966f-ebf8-42c7-838a-f15e847b03db None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:59.174281 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d966f-ebf8-42c7-838a-f15e847b03db None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:59.174578 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d966f-ebf8-42c7-838a-f15e847b03db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:59.184986 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d966f-ebf8-42c7-838a-f15e847b03db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:59.186380 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d966f-ebf8-42c7-838a-f15e847b03db None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:59.201213 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-559d966f-ebf8-42c7-838a-f15e847b03db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:59.201213 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:59.201213 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:59.201213 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:59.201213 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:59.201213 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:59.201213 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:59.201213 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:59.201213 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:59.201213 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:59.201213 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:59.201213 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:59.201213 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:15:59.201213 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:15:59.201213 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:15:59.201213 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:15:59.202138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:59.202138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:59.202138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:59.202138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:59.202138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:59.202138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:59.202138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:59.202138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:59.202138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:59.202138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:59.202138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:59.202138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:59.202138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:59.202138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:59.202138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:59.202138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:59.202138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:15:59.203410 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:15:59.203410 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:15:59.203410 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:59.203410 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 265/545] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:15:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:59.203906 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24096f5-73c3-44c9-8e95-10d760ed776d None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:59.204187 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24096f5-73c3-44c9-8e95-10d760ed776d None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:59.204457 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24096f5-73c3-44c9-8e95-10d760ed776d None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:59.209739 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 266/546] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:59.209739 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdcf59-64d1-49c8-923f-34510aa67918 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:59.209739 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdcf59-64d1-49c8-923f-34510aa67918 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:59.209739 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdcf59-64d1-49c8-923f-34510aa67918 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:59.209739 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 267/547] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:15:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:15:59.209739 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fdcf59-64d1-49c8-923f-34510aa67918 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:59.252002 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:59.281956 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eeca46-da6a-4c2b-809c-a4c599a6e5a2 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:59.314736 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de5aa0-b35c-4c53-86e5-18da76c6a346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:59.315348 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de5aa0-b35c-4c53-86e5-18da76c6a346 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:59.315623 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de5aa0-b35c-4c53-86e5-18da76c6a346 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:59.315889 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de5aa0-b35c-4c53-86e5-18da76c6a346 None admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:59.316519 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de5aa0-b35c-4c53-86e5-18da76c6a346 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:59.316816 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de5aa0-b35c-4c53-86e5-18da76c6a346 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:59.318203 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de5aa0-b35c-4c53-86e5-18da76c6a346 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:59.320998 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fdcf59-64d1-49c8-923f-34510aa67918 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:59.323302 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:59.330056 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 281/548] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:59.331307 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0f76b-95b5-4b1a-9b44-b404696429c3 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:59.331588 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0f76b-95b5-4b1a-9b44-b404696429c3 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:59.331858 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0f76b-95b5-4b1a-9b44-b404696429c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:59.349424 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-37a0f76b-95b5-4b1a-9b44-b404696429c3 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:59.350603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-37a0f76b-95b5-4b1a-9b44-b404696429c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:59.358419 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79ca7f9-5dac-4c46-8279-f6968f81f647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:59.359312 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ca7f9-5dac-4c46-8279-f6968f81f647 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:59.359839 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ca7f9-5dac-4c46-8279-f6968f81f647 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:59.360103 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ca7f9-5dac-4c46-8279-f6968f81f647 None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:59.360730 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79ca7f9-5dac-4c46-8279-f6968f81f647 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:59.361023 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79ca7f9-5dac-4c46-8279-f6968f81f647 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:59.362469 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79ca7f9-5dac-4c46-8279-f6968f81f647 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:59.378886 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 268/549] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:59.378886 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed4a72-49db-49dd-a9e2-cb41376ba875 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:59.378886 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed4a72-49db-49dd-a9e2-cb41376ba875 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:59.378886 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed4a72-49db-49dd-a9e2-cb41376ba875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:59.394937 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-43ed4a72-49db-49dd-a9e2-cb41376ba875 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:15:59.396204 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-43ed4a72-49db-49dd-a9e2-cb41376ba875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:15:59.426367 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 282/550] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:59.427293 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0f76b-95b5-4b1a-9b44-b404696429c3 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:59.480841 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 269/551] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:15:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 07:15:59.482146 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ed4a72-49db-49dd-a9e2-cb41376ba875 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:59.490572 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:59.544706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0f76b-95b5-4b1a-9b44-b404696429c3 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:59.587834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f229110-f0fa-4bc8-aa90-b5ab484fa152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:59.588810 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f229110-f0fa-4bc8-aa90-b5ab484fa152 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:59.589836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f229110-f0fa-4bc8-aa90-b5ab484fa152 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:59.590277 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f229110-f0fa-4bc8-aa90-b5ab484fa152 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:59.599847 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f229110-f0fa-4bc8-aa90-b5ab484fa152 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:59.601195 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:59.601501 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f229110-f0fa-4bc8-aa90-b5ab484fa152 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:59.614359 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ed4a72-49db-49dd-a9e2-cb41376ba875 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:59.615780 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-1f229110-f0fa-4bc8-aa90-b5ab484fa152 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 07:15:59.615780 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:59.615780 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:59.615780 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:59.615780 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:59.615780 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:59.615780 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:59.615780 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:59.615780 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:59.615780 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:59.615780 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:59.615780 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:59.615780 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:15:59.615780 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:15:59.615780 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:15:59.615780 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:15:59.616719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:59.616719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:59.616719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:15:59.616719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:59.616719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:15:59.616719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:15:59.616719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:59.616719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:59.616719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:59.616719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:59.616719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:59.616719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:59.616719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:59.616719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:59.616719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:59.616719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:59.616719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:59.616719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:59.617671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:59.617671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:59.617671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:15:59.617671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:15:59.617671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:15:59.617671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:15:59.617671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:15:59.617671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:15:59.617671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 07:15:59.617671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:59.617671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 283/552] 10.222.0.30 () {56 vars in 1117 bytes} [Wed Nov 2 07:15:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:59.617671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f229110-f0fa-4bc8-aa90-b5ab484fa152 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:59.654226 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d82f745-6b7d-41fb-aa18-fdad8a39ebca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:59.655272 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82f745-6b7d-41fb-aa18-fdad8a39ebca None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:59.655696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82f745-6b7d-41fb-aa18-fdad8a39ebca None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:59.656169 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82f745-6b7d-41fb-aa18-fdad8a39ebca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:59.663399 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d82f745-6b7d-41fb-aa18-fdad8a39ebca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:59.665114 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d82f745-6b7d-41fb-aa18-fdad8a39ebca None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:59.677349 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-2d82f745-6b7d-41fb-aa18-fdad8a39ebca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:59.677349 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:59.677349 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:59.677349 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:59.677349 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:59.677349 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:59.677349 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:59.677349 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:59.677349 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:59.677349 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:59.677349 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:59.677349 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:59.677349 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:15:59.677349 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:15:59.677349 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:15:59.678827 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:15:59.678827 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:59.678827 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:59.678827 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:59.678827 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:59.678827 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:59.678827 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:59.678827 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:59.678827 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:59.678827 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:59.678827 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:59.678827 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:59.678827 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:59.678827 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:59.678827 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:59.678827 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:59.678827 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:59.679710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:15:59.679710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:15:59.679710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:15:59.679710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:15:59.679710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:15:59.679710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:15:59.679710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:15:59.679710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:15:59.682100 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 270/553] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:15:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:59.683130 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d82f745-6b7d-41fb-aa18-fdad8a39ebca None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:59.730782 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f229110-f0fa-4bc8-aa90-b5ab484fa152 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:59.751948 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:59.765497 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a54d7-72f7-4887-b04d-eea0b436f6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:59.766123 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a54d7-72f7-4887-b04d-eea0b436f6bd None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:59.766392 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a54d7-72f7-4887-b04d-eea0b436f6bd None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:59.766672 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a54d7-72f7-4887-b04d-eea0b436f6bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:59.776462 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53a54d7-72f7-4887-b04d-eea0b436f6bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:15:59.782104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-c53a54d7-72f7-4887-b04d-eea0b436f6bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:59.782104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:15:59.782104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:15:59.782104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:15:59.782104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:15:59.782104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:15:59.782104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:15:59.782104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:15:59.782104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:15:59.782104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:15:59.782104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:15:59.782104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:15:59.782104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:15:59.782104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:15:59.782104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:15:59.782104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:15:59.783147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:15:59.783147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:15:59.783147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:15:59.783147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:15:59.783147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:15:59.783147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:15:59.783147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:15:59.783147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:15:59.783147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:15:59.783147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:15:59.783147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:15:59.783147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:15:59.783147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:15:59.783147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:15:59.783147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:15:59.783147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:15:59.783950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:15:59.783950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:15:59.783950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:15:59.783950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:15:59.783950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:15:59.783950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:15:59.783950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:15:59.783950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:15:59.783950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:15:59.783950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:15:59.783950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:15:59.783950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:15:59.783950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 284/554] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 07:15:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:15:59.783950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a54d7-72f7-4887-b04d-eea0b436f6bd None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:59.784860 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d82f745-6b7d-41fb-aa18-fdad8a39ebca None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:59.820745 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2007d9d-95de-4504-8bd7-70eb5e0d1100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:59.821354 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2007d9d-95de-4504-8bd7-70eb5e0d1100 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:59.821632 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2007d9d-95de-4504-8bd7-70eb5e0d1100 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:59.821899 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2007d9d-95de-4504-8bd7-70eb5e0d1100 None admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:59.822620 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2007d9d-95de-4504-8bd7-70eb5e0d1100 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:15:59.822958 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2007d9d-95de-4504-8bd7-70eb5e0d1100 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:59.824560 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2007d9d-95de-4504-8bd7-70eb5e0d1100 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:59.841833 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 271/555] 10.222.0.30 () {56 vars in 1149 bytes} [Wed Nov 2 07:15:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:15:59.842630 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2007d9d-95de-4504-8bd7-70eb5e0d1100 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:59.857904 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:15:59.884738 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a54d7-72f7-4887-b04d-eea0b436f6bd None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:59.918261 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef480cdb-4fa9-42fc-aa2f-4ca7588df0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:15:59.918925 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef480cdb-4fa9-42fc-aa2f-4ca7588df0d0 None admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:15:59.919198 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef480cdb-4fa9-42fc-aa2f-4ca7588df0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:15:59.919456 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef480cdb-4fa9-42fc-aa2f-4ca7588df0d0 None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:15:59.920171 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef480cdb-4fa9-42fc-aa2f-4ca7588df0d0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:15:59.921566 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef480cdb-4fa9-42fc-aa2f-4ca7588df0d0 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:15:59.937808 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 285/556] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Nov 2 07:15:59 2022] POST /identity/v3/services => generated 240 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:15:59.938597 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef480cdb-4fa9-42fc-aa2f-4ca7588df0d0 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:15:59.945072 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2007d9d-95de-4504-8bd7-70eb5e0d1100 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:15:59.986032 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:00.004313 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c25f8-dabc-451b-bb1e-4404672635a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:00.005063 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c25f8-dabc-451b-bb1e-4404672635a8 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:00.005192 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c25f8-dabc-451b-bb1e-4404672635a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:00.005373 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c25f8-dabc-451b-bb1e-4404672635a8 None admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:00.006056 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c25f8-dabc-451b-bb1e-4404672635a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:00.006356 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c25f8-dabc-451b-bb1e-4404672635a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:00.008404 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c25f8-dabc-451b-bb1e-4404672635a8 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:00.022644 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 272/557] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 07:15:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:00.023577 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c25f8-dabc-451b-bb1e-4404672635a8 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:00.064981 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef480cdb-4fa9-42fc-aa2f-4ca7588df0d0 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:00.105604 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0939fa7-b5cf-49fc-be59-48000b9c6e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:00.106231 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0939fa7-b5cf-49fc-be59-48000b9c6e12 None admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:00.106501 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0939fa7-b5cf-49fc-be59-48000b9c6e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:00.106795 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0939fa7-b5cf-49fc-be59-48000b9c6e12 None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:00.107448 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0939fa7-b5cf-49fc-be59-48000b9c6e12 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:00.107550 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:00.108839 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0939fa7-b5cf-49fc-be59-48000b9c6e12 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:00.126405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 286/558] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Nov 2 07:15:59 2022] POST /identity/v3/services => generated 251 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:00.127327 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0939fa7-b5cf-49fc-be59-48000b9c6e12 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:00.155677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c25f8-dabc-451b-bb1e-4404672635a8 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:00.212263 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d68a1-7fed-4396-92b9-0dad7cd2e375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:00.212993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d68a1-7fed-4396-92b9-0dad7cd2e375 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:00.213333 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d68a1-7fed-4396-92b9-0dad7cd2e375 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:00.213668 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d68a1-7fed-4396-92b9-0dad7cd2e375 None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:00.214475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6d68a1-7fed-4396-92b9-0dad7cd2e375 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:00.214938 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6d68a1-7fed-4396-92b9-0dad7cd2e375 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:00.216456 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6d68a1-7fed-4396-92b9-0dad7cd2e375 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:00.227805 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 273/559] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:00.228758 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d68a1-7fed-4396-92b9-0dad7cd2e375 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:00.272730 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:00.280381 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0939fa7-b5cf-49fc-be59-48000b9c6e12 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:00.335053 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e76abc-d288-4421-a62d-e8d975e207eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:00.336679 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e76abc-d288-4421-a62d-e8d975e207eb None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:00.336679 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e76abc-d288-4421-a62d-e8d975e207eb None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:00.336679 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e76abc-d288-4421-a62d-e8d975e207eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:00.349005 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e76abc-d288-4421-a62d-e8d975e207eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:00.350906 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e76abc-d288-4421-a62d-e8d975e207eb None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:00.359348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d68a1-7fed-4396-92b9-0dad7cd2e375 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:00.367326 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-b7e76abc-d288-4421-a62d-e8d975e207eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:00.367326 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:00.367326 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:00.367326 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:00.367326 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:00.367326 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:00.367326 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:00.367326 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:00.367326 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:00.367326 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:00.367326 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:00.367326 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:00.367326 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:16:00.367326 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:16:00.367326 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:16:00.367326 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:16:00.368274 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:00.368274 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:00.368274 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:00.368274 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:00.368274 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:00.368274 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:00.368274 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:00.368274 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:00.368274 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:00.368274 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:00.368274 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:00.368274 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:00.368274 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:00.368274 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:00.368274 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:00.368274 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:00.368274 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:16:00.369203 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:16:00.369203 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:00.369203 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:00.370767 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 287/560] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:00.374811 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e76abc-d288-4421-a62d-e8d975e207eb None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:00.386921 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:00.399499 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e569e55-ddcb-469d-a02e-618420d07d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:00.400167 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e569e55-ddcb-469d-a02e-618420d07d79 None admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:00.400462 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e569e55-ddcb-469d-a02e-618420d07d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:00.400710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e569e55-ddcb-469d-a02e-618420d07d79 None admin] PATH_INFO: `/v3/projects/25e2bc8417754f909b416b93d5b7afb7/users/836c4044aaf74de391219289b57d2956/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:00.403116 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e569e55-ddcb-469d-a02e-618420d07d79 None admin] RBAC: Authorizing `identity:create_grant(project_id=25e2bc8417754f909b416b93d5b7afb7, user_id=836c4044aaf74de391219289b57d2956, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:00.404897 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e569e55-ddcb-469d-a02e-618420d07d79 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:00.421410 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 274/561] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 07:16:00 2022] PUT /identity/v3/projects/25e2bc8417754f909b416b93d5b7afb7/users/836c4044aaf74de391219289b57d2956/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:16:00.422334 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e569e55-ddcb-469d-a02e-618420d07d79 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:00.542154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e76abc-d288-4421-a62d-e8d975e207eb None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:00.602147 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e569e55-ddcb-469d-a02e-618420d07d79 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:00.631260 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0c6f0-0032-47fb-94fd-8ade6c7e3135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:00.631913 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0c6f0-0032-47fb-94fd-8ade6c7e3135 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:00.632187 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0c6f0-0032-47fb-94fd-8ade6c7e3135 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:00.632456 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0c6f0-0032-47fb-94fd-8ade6c7e3135 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:00.643377 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0fb59-53fb-43d7-b286-4a5e1409cc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:00.644376 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0fb59-53fb-43d7-b286-4a5e1409cc7a None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:00.644845 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0fb59-53fb-43d7-b286-4a5e1409cc7a None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:00.645015 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f0c6f0-0032-47fb-94fd-8ade6c7e3135 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:00.645487 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0fb59-53fb-43d7-b286-4a5e1409cc7a None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:00.646336 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef0fb59-53fb-43d7-b286-4a5e1409cc7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:00.646440 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f0c6f0-0032-47fb-94fd-8ade6c7e3135 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:00.647044 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef0fb59-53fb-43d7-b286-4a5e1409cc7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:00.648821 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef0fb59-53fb-43d7-b286-4a5e1409cc7a None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:00.665224 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-b7f0c6f0-0032-47fb-94fd-8ade6c7e3135 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:00.665224 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:00.665224 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:00.665224 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:00.665224 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:00.665224 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:00.665224 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:00.665224 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:00.665224 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:00.665224 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:00.665224 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:00.665224 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:00.665224 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:16:00.665224 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:16:00.665224 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:16:00.665224 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:16:00.666214 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:00.666214 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:00.666214 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:00.666214 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:00.666214 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:00.666214 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:00.666214 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:00.666214 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:00.666214 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:00.666214 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:00.666214 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:00.666214 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:00.666214 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:00.666214 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:00.666214 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:00.666214 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:00.666214 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:16:00.667232 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:16:00.667232 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:00.667232 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:00.667232 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 288/563] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:00.668192 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 275/563] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:16:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:00.671778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0fb59-53fb-43d7-b286-4a5e1409cc7a None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:00.672730 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0c6f0-0032-47fb-94fd-8ade6c7e3135 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:00.678784 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:00.731315 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:00.798620 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0fb59-53fb-43d7-b286-4a5e1409cc7a None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:00.815289 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0c6f0-0032-47fb-94fd-8ade6c7e3135 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:00.844019 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8141b937-61cf-4ba4-8b1d-e2a5fda8c52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:00.845019 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8141b937-61cf-4ba4-8b1d-e2a5fda8c52e None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:00.845513 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8141b937-61cf-4ba4-8b1d-e2a5fda8c52e None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:00.845947 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8141b937-61cf-4ba4-8b1d-e2a5fda8c52e None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:00.846829 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8141b937-61cf-4ba4-8b1d-e2a5fda8c52e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:00.847396 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8141b937-61cf-4ba4-8b1d-e2a5fda8c52e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:00.849031 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8141b937-61cf-4ba4-8b1d-e2a5fda8c52e None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:00.856202 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291839df-4dbd-4817-93c3-40b4b7997d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:00.856927 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291839df-4dbd-4817-93c3-40b4b7997d44 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:00.857277 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291839df-4dbd-4817-93c3-40b4b7997d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:00.857608 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291839df-4dbd-4817-93c3-40b4b7997d44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:00.863936 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 276/564] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:16:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:00.873553 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291839df-4dbd-4817-93c3-40b4b7997d44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:00.873709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8141b937-61cf-4ba4-8b1d-e2a5fda8c52e None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:00.873813 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291839df-4dbd-4817-93c3-40b4b7997d44 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:00.892852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-291839df-4dbd-4817-93c3-40b4b7997d44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:16:00.892852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:00.892852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:00.892852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:00.892852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:00.892852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:00.892852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:00.892852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:00.892852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:00.892852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:00.892852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:00.892852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:00.892852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:16:00.892852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:16:00.892852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:16:00.893789 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:16:00.893789 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:00.893789 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:00.893789 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:00.893789 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:00.893789 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:00.893789 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:00.893789 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:00.893789 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:00.893789 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:00.893789 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:00.893789 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:00.893789 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:00.893789 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:00.893789 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:00.893789 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:00.893789 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:00.894681 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:16:00.894681 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:16:00.894681 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:16:00.894681 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:16:00.894681 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:16:00.894681 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:16:00.894681 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:16:00.894681 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:00.896267 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 289/565] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:16:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:00.905572 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291839df-4dbd-4817-93c3-40b4b7997d44 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:01.003948 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8141b937-61cf-4ba4-8b1d-e2a5fda8c52e None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:01.014275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:01.051292 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f11da7-2f03-4a14-8a1a-23f2667f4a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:01.051846 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f11da7-2f03-4a14-8a1a-23f2667f4a37 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:01.052122 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f11da7-2f03-4a14-8a1a-23f2667f4a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:01.052387 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f11da7-2f03-4a14-8a1a-23f2667f4a37 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:01.066628 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f11da7-2f03-4a14-8a1a-23f2667f4a37 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:01.067233 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:01.071049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f11da7-2f03-4a14-8a1a-23f2667f4a37 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:01.091439 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-c2f11da7-2f03-4a14-8a1a-23f2667f4a37 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:16:01.091439 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:01.091439 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:01.091439 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:01.091439 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:01.091439 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:01.091439 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:01.091439 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:01.091439 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:01.091439 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:01.091439 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:01.091439 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:01.091439 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:16:01.091439 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:16:01.091439 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:16:01.091439 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:16:01.092415 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:01.092415 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:01.092415 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:16:01.092415 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:16:01.092415 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:16:01.092415 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:16:01.092415 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:01.092415 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:01.092415 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:01.092415 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:01.092415 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:01.092415 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:01.092415 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:01.092415 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:01.092415 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:01.092415 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:01.092415 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:01.092415 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:01.093358 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:01.093358 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:01.093358 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:16:01.093358 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:16:01.093358 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:16:01.093358 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:16:01.093358 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:16:01.093358 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:16:01.093358 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:16:01.093358 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:01.098706 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 277/566] 10.222.0.30 () {56 vars in 1093 bytes} [Wed Nov 2 07:16:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:01.102022 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f11da7-2f03-4a14-8a1a-23f2667f4a37 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:01.138056 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291839df-4dbd-4817-93c3-40b4b7997d44 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:01.187773 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a835a1d-736d-4178-88b8-fe8da6c0f45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:01.188399 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a835a1d-736d-4178-88b8-fe8da6c0f45c None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:01.188677 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a835a1d-736d-4178-88b8-fe8da6c0f45c None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:01.188937 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a835a1d-736d-4178-88b8-fe8da6c0f45c None admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:01.189631 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a835a1d-736d-4178-88b8-fe8da6c0f45c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:01.189932 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a835a1d-736d-4178-88b8-fe8da6c0f45c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:01.191414 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a835a1d-736d-4178-88b8-fe8da6c0f45c None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:01.213255 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 290/567] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 07:16:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:01.219373 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a835a1d-736d-4178-88b8-fe8da6c0f45c None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:01.289514 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f11da7-2f03-4a14-8a1a-23f2667f4a37 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:01.356295 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4180c4c4-3529-4d7b-ba08-ee4817c5bef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:01.356929 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180c4c4-3529-4d7b-ba08-ee4817c5bef7 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:01.357208 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180c4c4-3529-4d7b-ba08-ee4817c5bef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:01.357478 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180c4c4-3529-4d7b-ba08-ee4817c5bef7 None admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:01.358210 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4180c4c4-3529-4d7b-ba08-ee4817c5bef7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:01.358567 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4180c4c4-3529-4d7b-ba08-ee4817c5bef7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:01.360041 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4180c4c4-3529-4d7b-ba08-ee4817c5bef7 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:01.374021 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a835a1d-736d-4178-88b8-fe8da6c0f45c None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:01.401228 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:01.405157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 278/568] 10.222.0.30 () {56 vars in 1133 bytes} [Wed Nov 2 07:16:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:01.410764 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4180c4c4-3529-4d7b-ba08-ee4817c5bef7 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:01.412627 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a81c7a-a6d8-48c8-86e3-e615d4fb1890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:01.413296 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a81c7a-a6d8-48c8-86e3-e615d4fb1890 None admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:01.413590 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a81c7a-a6d8-48c8-86e3-e615d4fb1890 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:01.414109 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a81c7a-a6d8-48c8-86e3-e615d4fb1890 None admin] PATH_INFO: `/v3/projects/25e2bc8417754f909b416b93d5b7afb7/users/364555ec385c4dbb91907fb9c5116d2a/roles/268fecadd98446f2b7d41755f3fe3649` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:01.431739 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a81c7a-a6d8-48c8-86e3-e615d4fb1890 None admin] RBAC: Authorizing `identity:create_grant(project_id=25e2bc8417754f909b416b93d5b7afb7, user_id=364555ec385c4dbb91907fb9c5116d2a, role_id=268fecadd98446f2b7d41755f3fe3649)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:01.433758 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a81c7a-a6d8-48c8-86e3-e615d4fb1890 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:01.458442 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:01.465552 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 291/569] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 07:16:01 2022] PUT /identity/v3/projects/25e2bc8417754f909b416b93d5b7afb7/users/364555ec385c4dbb91907fb9c5116d2a/roles/268fecadd98446f2b7d41755f3fe3649 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:16:01.546227 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4180c4c4-3529-4d7b-ba08-ee4817c5bef7 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:01.586013 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aedf03d-d710-4607-b98a-7f5551f2070b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:01.586767 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aedf03d-d710-4607-b98a-7f5551f2070b None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:01.587047 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aedf03d-d710-4607-b98a-7f5551f2070b None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:01.587364 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aedf03d-d710-4607-b98a-7f5551f2070b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:01.677003 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aedf03d-d710-4607-b98a-7f5551f2070b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:01.678471 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aedf03d-d710-4607-b98a-7f5551f2070b None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:01.705658 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:01.789978 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 279/570] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 07:16:01 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Nov 02 07:16:01.795656 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a81c7a-a6d8-48c8-86e3-e615d4fb1890 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:01.916105 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a81c7a-a6d8-48c8-86e3-e615d4fb1890 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:01.961794 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d06cf-bab4-4e6d-8b9d-005a6350f6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:01.962440 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d06cf-bab4-4e6d-8b9d-005a6350f6d6 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:01.962742 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d06cf-bab4-4e6d-8b9d-005a6350f6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:01.963015 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d06cf-bab4-4e6d-8b9d-005a6350f6d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:01.978319 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860d06cf-bab4-4e6d-8b9d-005a6350f6d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:01.979889 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860d06cf-bab4-4e6d-8b9d-005a6350f6d6 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:01.999756 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-860d06cf-bab4-4e6d-8b9d-005a6350f6d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:01.999756 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:01.999756 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:01.999756 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:01.999756 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:01.999756 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:01.999756 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:01.999756 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:01.999756 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:01.999756 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:01.999756 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:01.999756 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:01.999756 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:16:01.999756 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:16:01.999756 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:16:01.999756 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:16:02.001995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:02.001995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:02.001995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:02.001995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:02.001995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:02.001995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:02.001995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:02.001995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:02.001995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:02.001995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:02.001995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:02.001995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:02.001995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:02.001995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:02.001995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:02.001995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:02.001995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:16:02.004150 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:16:02.004150 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:02.004150 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:02.004150 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 292/571] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:02.006939 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aedf03d-d710-4607-b98a-7f5551f2070b None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:02.142486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aedf03d-d710-4607-b98a-7f5551f2070b None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:02.167394 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:02.180243 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08370b12-1bfe-4b82-8e1a-c63a6290232d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:02.181069 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08370b12-1bfe-4b82-8e1a-c63a6290232d None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:02.181483 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08370b12-1bfe-4b82-8e1a-c63a6290232d None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:02.181883 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08370b12-1bfe-4b82-8e1a-c63a6290232d None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:02.182735 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08370b12-1bfe-4b82-8e1a-c63a6290232d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:02.183261 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08370b12-1bfe-4b82-8e1a-c63a6290232d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:02.184806 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08370b12-1bfe-4b82-8e1a-c63a6290232d None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:02.197020 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 280/572] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:02.207968 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d06cf-bab4-4e6d-8b9d-005a6350f6d6 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:02.255290 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:02.341340 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d06cf-bab4-4e6d-8b9d-005a6350f6d6 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:02.380156 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a117545d-3aba-4ba0-aca7-0f9c09e20c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:02.380835 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117545d-3aba-4ba0-aca7-0f9c09e20c77 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:02.381078 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117545d-3aba-4ba0-aca7-0f9c09e20c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:02.381350 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117545d-3aba-4ba0-aca7-0f9c09e20c77 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:02.389552 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a117545d-3aba-4ba0-aca7-0f9c09e20c77 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:02.391162 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a117545d-3aba-4ba0-aca7-0f9c09e20c77 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:02.403944 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-a117545d-3aba-4ba0-aca7-0f9c09e20c77 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:16:02.403944 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:02.403944 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:02.403944 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:02.403944 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:02.403944 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:02.403944 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:02.403944 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:02.403944 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:02.403944 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:02.403944 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:02.403944 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:02.403944 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:16:02.403944 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:16:02.403944 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:16:02.404910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:16:02.404910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:02.404910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:02.404910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:02.404910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:02.404910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:02.404910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:02.404910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:02.404910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:02.404910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:02.404910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:02.404910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:02.404910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:02.404910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:02.404910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:02.404910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:02.404910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:02.406199 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:16:02.406199 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:16:02.406199 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:16:02.406199 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:16:02.406199 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:16:02.406199 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:16:02.406199 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:16:02.406199 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:02.406199 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 293/573] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:16:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:02.411178 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08370b12-1bfe-4b82-8e1a-c63a6290232d None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:02.511203 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08370b12-1bfe-4b82-8e1a-c63a6290232d None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:02.554430 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12a860-3be3-4421-a2fc-1dd7f418eb73 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:02.554796 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12a860-3be3-4421-a2fc-1dd7f418eb73 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:02.555067 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12a860-3be3-4421-a2fc-1dd7f418eb73 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:02.556578 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 294/574] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:16:02.557667 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1da39b-0085-48ab-b802-aea1e9e2dbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:02.558376 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1da39b-0085-48ab-b802-aea1e9e2dbd2 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:02.558681 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1da39b-0085-48ab-b802-aea1e9e2dbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:02.558952 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1da39b-0085-48ab-b802-aea1e9e2dbd2 None admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:02.559708 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1da39b-0085-48ab-b802-aea1e9e2dbd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:02.560017 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1da39b-0085-48ab-b802-aea1e9e2dbd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:02.561627 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1da39b-0085-48ab-b802-aea1e9e2dbd2 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:02.563351 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b4ecb-81ce-45ea-b849-b1e48cd7e620 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:02.563893 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b4ecb-81ce-45ea-b849-b1e48cd7e620 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:02.564361 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b4ecb-81ce-45ea-b849-b1e48cd7e620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:02.578417 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 281/575] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 07:16:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:02.585381 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1da39b-0085-48ab-b802-aea1e9e2dbd2 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:02.600367 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-8d3b4ecb-81ce-45ea-b849-b1e48cd7e620 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:02.601915 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-8d3b4ecb-81ce-45ea-b849-b1e48cd7e620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:02.680105 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:02.731688 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1da39b-0085-48ab-b802-aea1e9e2dbd2 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:02.758423 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 295/576] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:16:02.770822 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76235f6-6957-42a7-b88f-fd9c6c804075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:02.771886 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76235f6-6957-42a7-b88f-fd9c6c804075 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:02.772387 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76235f6-6957-42a7-b88f-fd9c6c804075 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:02.772851 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76235f6-6957-42a7-b88f-fd9c6c804075 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:02.773773 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76235f6-6957-42a7-b88f-fd9c6c804075 None admin] RBAC: Adding query filter params (role.id=1e35bb725eb8475ea63e2d9538ab67d5, scope.project.id=25e2bc8417754f909b416b93d5b7afb7, user.id=e6495385dccc4fa48c05b1feb957846c) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:02.774366 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76235f6-6957-42a7-b88f-fd9c6c804075 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:02.776053 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76235f6-6957-42a7-b88f-fd9c6c804075 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:02.783291 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 282/577] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 07:16:02 2022] GET /identity/v3/role_assignments?user.id=e6495385dccc4fa48c05b1feb957846c&scope.project.id=25e2bc8417754f909b416b93d5b7afb7&role.id=1e35bb725eb8475ea63e2d9538ab67d5 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:02.864192 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f34972-17bb-48d9-906e-b9cabf746b52 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:02.864532 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f34972-17bb-48d9-906e-b9cabf746b52 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:02.864739 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f34972-17bb-48d9-906e-b9cabf746b52 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:02.866262 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 296/578] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:02.883895 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0606d-7c3d-417e-954e-9127363283bd None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:02.884872 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0606d-7c3d-417e-954e-9127363283bd None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:02.885341 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0606d-7c3d-417e-954e-9127363283bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:02.911389 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-7cf0606d-7c3d-417e-954e-9127363283bd None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:02.912939 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-7cf0606d-7c3d-417e-954e-9127363283bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:03.001467 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 283/579] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:03.132114 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f34972-17bb-48d9-906e-b9cabf746b52 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:03.151560 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:03.205045 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929568c3-d124-4579-9cc7-c6835ec0998d None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:03.205409 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929568c3-d124-4579-9cc7-c6835ec0998d None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:03.205571 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929568c3-d124-4579-9cc7-c6835ec0998d None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:03.210793 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 284/580] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:16:03.212644 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc52b6d-65e8-457b-92a0-315d5486f3cd None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:03.212926 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc52b6d-65e8-457b-92a0-315d5486f3cd None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:03.213201 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc52b6d-65e8-457b-92a0-315d5486f3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:03.238541 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-3cc52b6d-65e8-457b-92a0-315d5486f3cd None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:03.240064 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-3cc52b6d-65e8-457b-92a0-315d5486f3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:03.254967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f34972-17bb-48d9-906e-b9cabf746b52 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:03.276713 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:03.298725 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f1e507-d9d5-494c-b3c7-0d2fbcbdaa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:03.299455 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f1e507-d9d5-494c-b3c7-0d2fbcbdaa0a None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:03.299836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f1e507-d9d5-494c-b3c7-0d2fbcbdaa0a None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:03.300399 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f1e507-d9d5-494c-b3c7-0d2fbcbdaa0a None admin] PATH_INFO: `/v3/services/network` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:03.301058 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f1e507-d9d5-494c-b3c7-0d2fbcbdaa0a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:03.310688 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f1e507-d9d5-494c-b3c7-0d2fbcbdaa0a None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:03.313016 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-20f1e507-d9d5-494c-b3c7-0d2fbcbdaa0a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:16:03.313016 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:03.313016 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:03.313016 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:03.313016 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:03.313016 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:03.313016 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:03.313016 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:03.313016 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:16:03.313016 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:03.313016 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:03.313016 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:03.313016 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:03.313016 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:03.313016 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:03.313016 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:03.313016 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:03.313967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:03.313967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:03.313967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:03.313967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:03.313967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:03.313967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:03.313967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:03.313967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:03.313967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:03.313967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:03.313967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:03.313967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:03.313967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:03.313967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:03.313967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:03.313967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:03.314844 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:03.314844 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:03.314844 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:03.314844 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:03.314844 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:03.314844 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:03.314844 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:03.314844 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:03.314844 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:03.314844 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:03.314844 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:03.314844 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:03.314844 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:16:03.314844 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:03.319112 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 297/581] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:16:03 2022] GET /identity/v3/services/network => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:03.320963 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f1e507-d9d5-494c-b3c7-0d2fbcbdaa0a None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:03.358791 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 285/582] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:16:03.359809 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc52b6d-65e8-457b-92a0-315d5486f3cd None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:03.416316 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:03.446319 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f1e507-d9d5-494c-b3c7-0d2fbcbdaa0a None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:03.485997 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc52b6d-65e8-457b-92a0-315d5486f3cd None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:03.509601 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415463c3-005c-4f67-9e92-f184e477bd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:03.510218 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415463c3-005c-4f67-9e92-f184e477bd88 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:03.510750 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415463c3-005c-4f67-9e92-f184e477bd88 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:03.511012 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415463c3-005c-4f67-9e92-f184e477bd88 None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:03.511645 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415463c3-005c-4f67-9e92-f184e477bd88 None admin] RBAC: Adding query filter params (name=network) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:03.511951 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415463c3-005c-4f67-9e92-f184e477bd88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:03.513340 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415463c3-005c-4f67-9e92-f184e477bd88 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:03.522484 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a951ec87-03eb-405b-9275-0e8f7a267bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:03.523154 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a951ec87-03eb-405b-9275-0e8f7a267bfa None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:03.523429 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a951ec87-03eb-405b-9275-0e8f7a267bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:03.523695 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a951ec87-03eb-405b-9275-0e8f7a267bfa None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:03.524458 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a951ec87-03eb-405b-9275-0e8f7a267bfa None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:03.525129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 298/583] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Nov 2 07:16:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:03.530990 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a951ec87-03eb-405b-9275-0e8f7a267bfa None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:03.542494 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-a951ec87-03eb-405b-9275-0e8f7a267bfa None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:16:03.542494 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:03.542494 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:03.542494 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:03.542494 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:03.542494 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:03.542494 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:03.542494 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:03.542494 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:16:03.542494 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:03.542494 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:03.542494 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:03.542494 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:03.542494 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:03.542494 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:03.542494 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:03.542494 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:03.543575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:03.543575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:03.543575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:03.543575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:03.543575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:03.543575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:03.543575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:03.543575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:03.543575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:03.543575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:03.543575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:03.543575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:03.543575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:03.543575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:03.543575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:03.543575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:03.544450 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:03.544450 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:03.544450 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:03.544450 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:03.544450 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:03.544450 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:03.544450 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:03.544450 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:03.544450 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:03.544450 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:03.544450 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:03.544450 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:03.544450 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:16:03.544450 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:03.545201 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415463c3-005c-4f67-9e92-f184e477bd88 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:03.545273 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 286/584] 10.222.0.30 () {56 vars in 1129 bytes} [Wed Nov 2 07:16:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:03.556034 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a951ec87-03eb-405b-9275-0e8f7a267bfa None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:03.590481 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:03.665305 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a951ec87-03eb-405b-9275-0e8f7a267bfa None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:03.683767 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415463c3-005c-4f67-9e92-f184e477bd88 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:03.718907 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b64d4-a42a-469b-a052-edfd3b9f0506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:03.719764 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b64d4-a42a-469b-a052-edfd3b9f0506 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:03.720046 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b64d4-a42a-469b-a052-edfd3b9f0506 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:03.720318 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b64d4-a42a-469b-a052-edfd3b9f0506 None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:03.721012 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573b64d4-a42a-469b-a052-edfd3b9f0506 None admin] RBAC: Adding query filter params (type=network) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:03.721249 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573b64d4-a42a-469b-a052-edfd3b9f0506 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:03.730835 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573b64d4-a42a-469b-a052-edfd3b9f0506 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:03.730835 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 299/585] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Nov 2 07:16:03 2022] GET /identity/v3/services?type=network => generated 350 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:03.733746 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b64d4-a42a-469b-a052-edfd3b9f0506 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:03.751578 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c47669-85ff-449c-b50c-3a9317d96dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:03.752194 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c47669-85ff-449c-b50c-3a9317d96dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:03.752470 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c47669-85ff-449c-b50c-3a9317d96dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:03.752738 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c47669-85ff-449c-b50c-3a9317d96dd5 None admin] PATH_INFO: `/v3/services` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:03.753369 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c47669-85ff-449c-b50c-3a9317d96dd5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:03.753669 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c47669-85ff-449c-b50c-3a9317d96dd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:03.756252 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c47669-85ff-449c-b50c-3a9317d96dd5 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:03.763020 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:03.767436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 287/586] 10.222.0.30 () {56 vars in 1124 bytes} [Wed Nov 2 07:16:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:03.777086 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c47669-85ff-449c-b50c-3a9317d96dd5 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:03.863288 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b64d4-a42a-469b-a052-edfd3b9f0506 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:03.898453 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f918cc-b8d8-473d-b9ee-22cc8e64e097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:03.899131 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f918cc-b8d8-473d-b9ee-22cc8e64e097 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:03.899406 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f918cc-b8d8-473d-b9ee-22cc8e64e097 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:03.899672 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f918cc-b8d8-473d-b9ee-22cc8e64e097 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:03.900362 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f918cc-b8d8-473d-b9ee-22cc8e64e097 None admin] RBAC: Adding query filter params (interface=public, service_id=290cf213d6c34786964cc1ed76f2ac9f, region_id=RegionOne) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:03.900671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f918cc-b8d8-473d-b9ee-22cc8e64e097 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:03.902104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f918cc-b8d8-473d-b9ee-22cc8e64e097 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:03.904022 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:03.911433 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c47669-85ff-449c-b50c-3a9317d96dd5 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:03.915374 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 300/587] 10.222.0.30 () {56 vars in 1251 bytes} [Wed Nov 2 07:16:03 2022] GET /identity/v3/endpoints?service_id=290cf213d6c34786964cc1ed76f2ac9f&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:03.916708 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f918cc-b8d8-473d-b9ee-22cc8e64e097 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:03.962732 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648de5c3-7900-4f98-9cd3-6d02a6383739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:03.963802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648de5c3-7900-4f98-9cd3-6d02a6383739 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:03.964146 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648de5c3-7900-4f98-9cd3-6d02a6383739 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:03.964565 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648de5c3-7900-4f98-9cd3-6d02a6383739 None admin] PATH_INFO: `/v3/services` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:03.965384 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648de5c3-7900-4f98-9cd3-6d02a6383739 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:03.965689 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648de5c3-7900-4f98-9cd3-6d02a6383739 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:03.967938 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648de5c3-7900-4f98-9cd3-6d02a6383739 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:03.978819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 288/588] 10.222.0.30 () {56 vars in 1124 bytes} [Wed Nov 2 07:16:03 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:03.979767 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648de5c3-7900-4f98-9cd3-6d02a6383739 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:03.997397 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:04.030544 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f918cc-b8d8-473d-b9ee-22cc8e64e097 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:04.067472 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fedc30-c6fe-485c-81ab-4c06f5d51b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:04.069576 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fedc30-c6fe-485c-81ab-4c06f5d51b9c None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:04.069961 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fedc30-c6fe-485c-81ab-4c06f5d51b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:04.070318 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fedc30-c6fe-485c-81ab-4c06f5d51b9c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:04.082027 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fedc30-c6fe-485c-81ab-4c06f5d51b9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:04.087769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-66fedc30-c6fe-485c-81ab-4c06f5d51b9c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:04.087769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:04.087769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:04.087769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:04.087769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:04.087769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:04.087769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:04.087769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:04.087769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:04.087769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:04.087769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:04.087769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:04.087769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:04.087769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:04.087769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:04.087769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:04.090180 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:04.090180 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:04.090180 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:04.090180 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:04.090180 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:04.090180 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:04.090180 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:04.090180 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:04.090180 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:04.090180 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:04.090180 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:04.090180 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:04.090180 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:04.090180 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:04.090180 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:04.090180 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:04.092150 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:04.092150 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:04.092150 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:04.092150 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:04.092150 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:04.092150 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:04.092150 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:04.092150 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:04.092150 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:04.092150 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:04.092150 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:04.092150 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:04.098697 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 301/589] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 07:16:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:04.098697 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fedc30-c6fe-485c-81ab-4c06f5d51b9c None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:04.112181 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648de5c3-7900-4f98-9cd3-6d02a6383739 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:04.158778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a226755-f643-4efd-ba0c-c7678838ae41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:04.159552 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a226755-f643-4efd-ba0c-c7678838ae41 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:04.159831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a226755-f643-4efd-ba0c-c7678838ae41 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:04.160387 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a226755-f643-4efd-ba0c-c7678838ae41 None admin] PATH_INFO: `/v3/services` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:04.161325 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a226755-f643-4efd-ba0c-c7678838ae41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:04.162913 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a226755-f643-4efd-ba0c-c7678838ae41 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:04.170098 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 289/590] 10.222.0.30 () {56 vars in 1087 bytes} [Wed Nov 2 07:16:03 2022] GET /identity/v3/services => generated 772 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:04.170938 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a226755-f643-4efd-ba0c-c7678838ae41 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:04.201437 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:04.233002 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fedc30-c6fe-485c-81ab-4c06f5d51b9c None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:04.268121 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:04.273844 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3365c-22c2-4443-89c3-9e360e309c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:04.274853 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3365c-22c2-4443-89c3-9e360e309c5c None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:04.275473 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3365c-22c2-4443-89c3-9e360e309c5c None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:04.276059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3365c-22c2-4443-89c3-9e360e309c5c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:04.277220 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd3365c-22c2-4443-89c3-9e360e309c5c None admin] RBAC: Adding query filter params (interface=public, service_id=0e8dc46190d74f47a64a24764655f54d, region_id=RegionOne) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:04.277715 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd3365c-22c2-4443-89c3-9e360e309c5c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:04.279720 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd3365c-22c2-4443-89c3-9e360e309c5c None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:04.286145 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 302/591] 10.222.0.30 () {56 vars in 1251 bytes} [Wed Nov 2 07:16:04 2022] GET /identity/v3/endpoints?service_id=0e8dc46190d74f47a64a24764655f54d&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:04.287891 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa7e35-de5b-4917-9969-1e82704641cb None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:04.288321 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa7e35-de5b-4917-9969-1e82704641cb None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:04.288735 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa7e35-de5b-4917-9969-1e82704641cb None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:04.290583 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 303/592] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:04.294038 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a226755-f643-4efd-ba0c-c7678838ae41 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:04.296327 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85ea80-1ee0-4515-9977-474da540410c None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:04.297733 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85ea80-1ee0-4515-9977-474da540410c None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:04.298188 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85ea80-1ee0-4515-9977-474da540410c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:04.323184 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-4b85ea80-1ee0-4515-9977-474da540410c None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:04.325439 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-4b85ea80-1ee0-4515-9977-474da540410c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:04.333916 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb1d365-b0f6-42c7-9a6b-50b65848b164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:04.334582 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb1d365-b0f6-42c7-9a6b-50b65848b164 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:04.334903 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb1d365-b0f6-42c7-9a6b-50b65848b164 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:04.335172 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb1d365-b0f6-42c7-9a6b-50b65848b164 None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:04.335843 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb1d365-b0f6-42c7-9a6b-50b65848b164 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:04.336183 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb1d365-b0f6-42c7-9a6b-50b65848b164 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:04.337801 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb1d365-b0f6-42c7-9a6b-50b65848b164 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:04.353563 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 290/593] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:04.355264 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb1d365-b0f6-42c7-9a6b-50b65848b164 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:04.417870 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 304/594] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:04.419501 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85ea80-1ee0-4515-9977-474da540410c None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:04.459556 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:04.480242 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb1d365-b0f6-42c7-9a6b-50b65848b164 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:04.528634 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95f848-689b-4faa-90a3-22858efed80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:04.529163 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95f848-689b-4faa-90a3-22858efed80f None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:04.529439 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95f848-689b-4faa-90a3-22858efed80f None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:04.529697 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95f848-689b-4faa-90a3-22858efed80f None admin] PATH_INFO: `/v3/services` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:04.530296 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea95f848-689b-4faa-90a3-22858efed80f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:04.531721 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea95f848-689b-4faa-90a3-22858efed80f None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:04.539501 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85ea80-1ee0-4515-9977-474da540410c None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:04.546175 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 291/595] 10.222.0.30 () {56 vars in 1087 bytes} [Wed Nov 2 07:16:04 2022] GET /identity/v3/services => generated 772 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:04.594974 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:04.611536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9184eb-c9a6-4789-a4d5-5a2d8e5db12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:04.612218 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9184eb-c9a6-4789-a4d5-5a2d8e5db12a None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:04.612491 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9184eb-c9a6-4789-a4d5-5a2d8e5db12a None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:04.612768 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9184eb-c9a6-4789-a4d5-5a2d8e5db12a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:04.624326 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9184eb-c9a6-4789-a4d5-5a2d8e5db12a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:04.625798 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9184eb-c9a6-4789-a4d5-5a2d8e5db12a None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:04.653995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-0a9184eb-c9a6-4789-a4d5-5a2d8e5db12a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:04.653995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:04.653995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:04.653995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:04.653995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:04.653995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:04.653995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:04.653995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:04.653995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:04.653995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:04.653995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:04.653995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:04.653995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:16:04.653995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:16:04.653995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:16:04.653995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:16:04.655040 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:04.655040 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:04.655040 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:04.655040 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:04.655040 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:04.655040 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:04.655040 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:04.655040 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:04.655040 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:04.655040 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:04.655040 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:04.655040 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:04.655040 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:04.655040 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:04.655040 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:04.655040 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:04.655040 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:16:04.655949 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:16:04.655949 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:04.655949 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:04.655949 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 305/596] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:04.661066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95f848-689b-4faa-90a3-22858efed80f None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:04.725683 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9184eb-c9a6-4789-a4d5-5a2d8e5db12a None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:04.792982 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95f848-689b-4faa-90a3-22858efed80f None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:04.864800 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f7ffde-184f-4610-bee9-1cb9f9d1bc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:04.865433 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7ffde-184f-4610-bee9-1cb9f9d1bc93 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:04.865713 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7ffde-184f-4610-bee9-1cb9f9d1bc93 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:04.865987 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7ffde-184f-4610-bee9-1cb9f9d1bc93 None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:04.866684 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f7ffde-184f-4610-bee9-1cb9f9d1bc93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:04.866983 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f7ffde-184f-4610-bee9-1cb9f9d1bc93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:04.868413 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f7ffde-184f-4610-bee9-1cb9f9d1bc93 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:04.891831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 292/597] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:16:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:04.897215 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f7ffde-184f-4610-bee9-1cb9f9d1bc93 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:04.914693 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:04.934684 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9184eb-c9a6-4789-a4d5-5a2d8e5db12a None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:04.984374 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d8e60-7e86-4544-bdd2-5a7bc710aaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:04.985035 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d8e60-7e86-4544-bdd2-5a7bc710aaad None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:04.986086 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d8e60-7e86-4544-bdd2-5a7bc710aaad None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:04.986086 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d8e60-7e86-4544-bdd2-5a7bc710aaad None admin] PATH_INFO: `/v3/roles/service` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:04.999501 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55d8e60-7e86-4544-bdd2-5a7bc710aaad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:05.005612 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-f55d8e60-7e86-4544-bdd2-5a7bc710aaad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:16:05.005612 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:05.005612 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:05.005612 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:05.005612 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:05.005612 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:05.005612 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:05.005612 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:05.005612 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:05.005612 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:05.005612 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:05.005612 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:05.005612 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:05.005612 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:05.005612 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:05.005612 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:05.007761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:05.007761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:05.007761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:05.007761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:05.007761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:05.007761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:05.007761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:05.007761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:05.007761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:05.007761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:05.007761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:05.007761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:05.007761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:05.007761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:05.007761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:05.007761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:05.009669 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:05.009669 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:05.009669 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:05.009669 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:05.009669 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:05.009669 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:05.009669 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:05.009669 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:05.009669 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:05.009669 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:05.009669 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:16:05.009669 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:05.009669 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 306/598] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Nov 2 07:16:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:05.016309 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d8e60-7e86-4544-bdd2-5a7bc710aaad None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:05.027502 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f7ffde-184f-4610-bee9-1cb9f9d1bc93 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:05.070837 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d270e71-ec2d-41c3-abe3-ceaf83401eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:05.071390 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d270e71-ec2d-41c3-abe3-ceaf83401eaf None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:05.071668 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d270e71-ec2d-41c3-abe3-ceaf83401eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:05.071932 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d270e71-ec2d-41c3-abe3-ceaf83401eaf None admin] PATH_INFO: `/v3/users/swift` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:05.090134 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d270e71-ec2d-41c3-abe3-ceaf83401eaf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:05.091680 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d270e71-ec2d-41c3-abe3-ceaf83401eaf None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:05.098230 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:05.114173 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-1d270e71-ec2d-41c3-abe3-ceaf83401eaf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Nov 02 07:16:05.114173 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:05.114173 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:05.114173 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:05.114173 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:05.114173 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:05.114173 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:05.114173 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:05.114173 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:05.114173 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:05.114173 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:05.114173 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:05.114173 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:16:05.114173 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:16:05.114173 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:16:05.114173 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:16:05.115236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:05.115236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:05.115236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:16:05.115236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:16:05.115236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:16:05.115236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:16:05.115236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:05.115236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:05.115236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:05.115236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:05.115236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:05.115236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:05.115236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:05.115236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:05.115236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:05.115236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:05.115236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:05.115236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:05.116184 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:05.116184 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:05.116184 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:16:05.116184 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:16:05.116184 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:16:05.116184 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:16:05.116184 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:16:05.116184 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:16:05.116184 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Nov 02 07:16:05.116184 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:05.116184 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 293/599] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 07:16:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:05.123246 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d270e71-ec2d-41c3-abe3-ceaf83401eaf None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:05.205635 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:05.231577 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d8e60-7e86-4544-bdd2-5a7bc710aaad None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:05.242699 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d270e71-ec2d-41c3-abe3-ceaf83401eaf None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:05.282512 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f2c69-e900-4ef5-be04-b335bee0a92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:05.283153 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f2c69-e900-4ef5-be04-b335bee0a92e None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:05.283426 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f2c69-e900-4ef5-be04-b335bee0a92e None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:05.283693 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f2c69-e900-4ef5-be04-b335bee0a92e None admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:05.284422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624f2c69-e900-4ef5-be04-b335bee0a92e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:05.284710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624f2c69-e900-4ef5-be04-b335bee0a92e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:05.286308 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624f2c69-e900-4ef5-be04-b335bee0a92e None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:05.306107 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 294/600] 10.222.0.30 () {56 vars in 1135 bytes} [Wed Nov 2 07:16:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:05.312205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f2c69-e900-4ef5-be04-b335bee0a92e None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:05.316755 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dbb7b3-4a51-4b16-ae9f-b89d61bf9b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:05.317645 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbb7b3-4a51-4b16-ae9f-b89d61bf9b1c None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:05.318140 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbb7b3-4a51-4b16-ae9f-b89d61bf9b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:05.318609 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbb7b3-4a51-4b16-ae9f-b89d61bf9b1c None admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:05.319491 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dbb7b3-4a51-4b16-ae9f-b89d61bf9b1c None admin] RBAC: Adding query filter params (name=service) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:05.320005 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dbb7b3-4a51-4b16-ae9f-b89d61bf9b1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:05.321656 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dbb7b3-4a51-4b16-ae9f-b89d61bf9b1c None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:05.342490 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 307/601] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Nov 2 07:16:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:05.346805 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dbb7b3-4a51-4b16-ae9f-b89d61bf9b1c None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:05.448305 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f2c69-e900-4ef5-be04-b335bee0a92e None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:05.478578 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:05.485944 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5907f666-cd74-41e3-beb3-be5797288ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:05.487050 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5907f666-cd74-41e3-beb3-be5797288ceb None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:05.487426 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5907f666-cd74-41e3-beb3-be5797288ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:05.487792 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5907f666-cd74-41e3-beb3-be5797288ceb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:05.515840 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:05.519761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dbb7b3-4a51-4b16-ae9f-b89d61bf9b1c None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:05.556255 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd99afc-d9bf-4245-a73b-e599f5db24ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:05.557212 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd99afc-d9bf-4245-a73b-e599f5db24ac None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:05.557633 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd99afc-d9bf-4245-a73b-e599f5db24ac None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:05.558065 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd99afc-d9bf-4245-a73b-e599f5db24ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:05.562543 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5907f666-cd74-41e3-beb3-be5797288ceb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:05.564128 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5907f666-cd74-41e3-beb3-be5797288ceb None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:05.570200 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd99afc-d9bf-4245-a73b-e599f5db24ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:05.571929 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd99afc-d9bf-4245-a73b-e599f5db24ac None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:05.587744 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-6bd99afc-d9bf-4245-a73b-e599f5db24ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:05.587744 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:05.587744 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:05.587744 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:05.587744 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:05.587744 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:05.587744 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:05.587744 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:05.587744 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:05.587744 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:05.587744 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:05.587744 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:05.587744 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:16:05.587744 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:16:05.587744 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:16:05.587744 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:16:05.588831 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:05.588831 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:05.588831 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:05.588831 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:05.588831 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:05.588831 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:05.588831 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:05.588831 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:05.588831 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:05.588831 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:05.588831 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:05.588831 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:05.588831 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:05.588831 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:05.588831 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:05.588831 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:05.588831 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:16:05.589733 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:16:05.589733 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:05.589733 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:05.597191 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 308/602] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:05.598763 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe205a6-c0c6-4028-b56b-bf734f844c6a None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:05.599274 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe205a6-c0c6-4028-b56b-bf734f844c6a None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:05.599726 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe205a6-c0c6-4028-b56b-bf734f844c6a None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:05.601524 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 309/603] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:05.602392 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe205a6-c0c6-4028-b56b-bf734f844c6a None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:05.673486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 295/604] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 07:16:05 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:16:05.675043 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a31b8-c85d-43b1-bfd6-871cc9101746 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:05.675478 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a31b8-c85d-43b1-bfd6-871cc9101746 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:05.675921 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a31b8-c85d-43b1-bfd6-871cc9101746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:05.697725 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-835a31b8-c85d-43b1-bfd6-871cc9101746 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:05.699405 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-835a31b8-c85d-43b1-bfd6-871cc9101746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:05.723607 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe205a6-c0c6-4028-b56b-bf734f844c6a None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:05.751043 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:05.759688 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d25aa-0604-4c3f-898d-b2946cbc3651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:05.760569 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d25aa-0604-4c3f-898d-b2946cbc3651 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:05.761007 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d25aa-0604-4c3f-898d-b2946cbc3651 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:05.761516 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d25aa-0604-4c3f-898d-b2946cbc3651 None admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:05.762382 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d25aa-0604-4c3f-898d-b2946cbc3651 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:05.762977 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d25aa-0604-4c3f-898d-b2946cbc3651 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:05.764595 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d25aa-0604-4c3f-898d-b2946cbc3651 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:05.784573 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 296/605] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:05.784573 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a31b8-c85d-43b1-bfd6-871cc9101746 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:05.785837 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 310/606] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:05.791010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d25aa-0604-4c3f-898d-b2946cbc3651 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:05.834112 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:05.973803 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a31b8-c85d-43b1-bfd6-871cc9101746 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:05.993102 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d25aa-0604-4c3f-898d-b2946cbc3651 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:06.012724 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649cf4a4-faa7-4659-a80b-cf7940d25a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:06.013438 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649cf4a4-faa7-4659-a80b-cf7940d25a18 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:06.013775 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649cf4a4-faa7-4659-a80b-cf7940d25a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:06.014105 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649cf4a4-faa7-4659-a80b-cf7940d25a18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:06.030677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649cf4a4-faa7-4659-a80b-cf7940d25a18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:06.030677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649cf4a4-faa7-4659-a80b-cf7940d25a18 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:06.044302 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-649cf4a4-faa7-4659-a80b-cf7940d25a18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:06.044302 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:06.044302 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:06.044302 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:06.044302 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:06.044302 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:06.044302 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:06.044302 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:06.044302 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:06.044302 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:06.044302 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:06.044302 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:06.044302 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:16:06.044302 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:16:06.044302 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:16:06.044302 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:16:06.045536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:06.045536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:06.045536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:06.045536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:06.045536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:06.045536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:06.045536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:06.045536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:06.045536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:06.045536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:06.045536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:06.045536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:06.045536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:06.045536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:06.045536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:06.045536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:06.045536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:16:06.046754 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:16:06.046754 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:06.046754 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:06.054279 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 297/607] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:06.054279 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649cf4a4-faa7-4659-a80b-cf7940d25a18 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:06.071708 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008618f-68c9-48dc-9465-3029fcbf68e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:06.072742 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008618f-68c9-48dc-9465-3029fcbf68e7 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:06.073265 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008618f-68c9-48dc-9465-3029fcbf68e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:06.073725 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008618f-68c9-48dc-9465-3029fcbf68e7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:06.076072 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:06.094344 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f008618f-68c9-48dc-9465-3029fcbf68e7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:06.096176 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f008618f-68c9-48dc-9465-3029fcbf68e7 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:06.122706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-f008618f-68c9-48dc-9465-3029fcbf68e7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 07:16:06.122706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:06.122706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:06.122706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:06.122706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:06.122706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:06.122706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:06.122706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:06.122706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:06.122706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:06.122706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:06.122706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:06.122706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:16:06.122706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:16:06.122706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:16:06.122706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:16:06.123721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:06.123721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:06.123721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:16:06.123721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:16:06.123721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:16:06.123721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:16:06.123721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:06.123721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:06.123721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:06.123721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:06.123721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:06.123721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:06.123721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:06.123721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:06.123721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:06.123721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:06.123721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:06.123721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:06.124761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:06.124761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:06.124761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:16:06.124761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:16:06.124761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:16:06.124761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:16:06.124761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:16:06.124761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:16:06.124761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 07:16:06.124761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:06.131732 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 311/608] 10.222.0.30 () {56 vars in 1117 bytes} [Wed Nov 2 07:16:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:06.132829 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008618f-68c9-48dc-9465-3029fcbf68e7 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:06.202424 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649cf4a4-faa7-4659-a80b-cf7940d25a18 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:06.219067 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:06.238863 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008618f-68c9-48dc-9465-3029fcbf68e7 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:06.247199 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a490e42-86f3-4e66-a746-ba2500f0967a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:06.247824 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a490e42-86f3-4e66-a746-ba2500f0967a None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:06.248102 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a490e42-86f3-4e66-a746-ba2500f0967a None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:06.248372 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a490e42-86f3-4e66-a746-ba2500f0967a None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:06.249528 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a490e42-86f3-4e66-a746-ba2500f0967a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:06.249837 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a490e42-86f3-4e66-a746-ba2500f0967a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:06.251784 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a490e42-86f3-4e66-a746-ba2500f0967a None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:06.270763 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 298/609] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:16:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:06.271664 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a490e42-86f3-4e66-a746-ba2500f0967a None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:06.281410 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ef0b8-bfc0-4cb2-9340-9810a55d1282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:06.282098 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ef0b8-bfc0-4cb2-9340-9810a55d1282 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:06.282378 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ef0b8-bfc0-4cb2-9340-9810a55d1282 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:06.283057 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ef0b8-bfc0-4cb2-9340-9810a55d1282 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:06.283393 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ef0b8-bfc0-4cb2-9340-9810a55d1282 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:06.284781 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ef0b8-bfc0-4cb2-9340-9810a55d1282 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:06.294294 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-8e4ef0b8-bfc0-4cb2-9340-9810a55d1282 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:16:06.294294 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:06.294294 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:06.294294 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:06.294294 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:06.294294 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:06.294294 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:06.294294 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:06.294294 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:16:06.294294 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:06.294294 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:06.294294 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:06.294294 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:06.294294 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:06.294294 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:06.294294 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:06.295308 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:06.295308 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:06.295308 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:06.295308 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:06.295308 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:06.295308 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:06.295308 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:06.295308 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:06.295308 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:06.295308 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:06.295308 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:06.295308 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:06.295308 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:06.295308 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:06.295308 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:06.295308 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:06.295308 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:06.296753 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:06.296753 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:06.296753 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:06.296753 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:06.296753 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:06.296753 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:06.296753 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:06.296753 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:06.296753 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:06.296753 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:06.296753 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:06.296753 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:06.296753 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:16:06.296753 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:06.296753 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 312/610] 10.222.0.30 () {56 vars in 1132 bytes} [Wed Nov 2 07:16:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:06.297552 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ef0b8-bfc0-4cb2-9340-9810a55d1282 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:06.355527 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:06.394265 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a490e42-86f3-4e66-a746-ba2500f0967a None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:06.405190 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ef0b8-bfc0-4cb2-9340-9810a55d1282 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:06.437095 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd0e65-0c21-4b3e-8e51-5caf70c6b86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:06.438078 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd0e65-0c21-4b3e-8e51-5caf70c6b86e None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:06.438559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd0e65-0c21-4b3e-8e51-5caf70c6b86e None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:06.439128 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd0e65-0c21-4b3e-8e51-5caf70c6b86e None admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:06.440196 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fd0e65-0c21-4b3e-8e51-5caf70c6b86e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:06.440686 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fd0e65-0c21-4b3e-8e51-5caf70c6b86e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:06.441967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2f760-5a90-4aff-853c-c6d145648647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:06.442620 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fd0e65-0c21-4b3e-8e51-5caf70c6b86e None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:06.442759 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2f760-5a90-4aff-853c-c6d145648647 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:06.443046 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2f760-5a90-4aff-853c-c6d145648647 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:06.443380 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2f760-5a90-4aff-853c-c6d145648647 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:06.446554 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:06.451734 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f2f760-5a90-4aff-853c-c6d145648647 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:06.453314 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f2f760-5a90-4aff-853c-c6d145648647 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:06.465484 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 299/611] 10.222.0.30 () {56 vars in 1149 bytes} [Wed Nov 2 07:16:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:06.466681 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd0e65-0c21-4b3e-8e51-5caf70c6b86e None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:06.471671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-08f2f760-5a90-4aff-853c-c6d145648647 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:16:06.471671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:06.471671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:06.471671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:06.471671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:06.471671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:06.471671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:06.471671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:06.471671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:06.471671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:06.471671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:06.471671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:06.471671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:16:06.471671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:16:06.471671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:16:06.472665 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:16:06.472665 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:06.472665 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:06.472665 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:06.472665 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:06.472665 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:06.472665 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:06.472665 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:06.472665 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:06.472665 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:06.472665 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:06.472665 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:06.472665 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:06.472665 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:06.472665 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:06.472665 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:06.472665 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:06.473719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:16:06.473719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:16:06.473719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:16:06.473719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:16:06.473719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:16:06.473719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:16:06.473719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:16:06.473719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:06.473719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 313/612] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:16:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:06.474692 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2f760-5a90-4aff-853c-c6d145648647 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:06.591225 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2f760-5a90-4aff-853c-c6d145648647 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:06.593674 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd0e65-0c21-4b3e-8e51-5caf70c6b86e None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:06.624613 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:06.631328 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672883b-ec98-41eb-a86a-f6ec5d486cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:06.631951 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672883b-ec98-41eb-a86a-f6ec5d486cbf None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:06.632227 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672883b-ec98-41eb-a86a-f6ec5d486cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:06.632501 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672883b-ec98-41eb-a86a-f6ec5d486cbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:06.633765 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0914e38-d591-4992-b16b-68d2be4f756f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:06.634540 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0914e38-d591-4992-b16b-68d2be4f756f None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:06.635300 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0914e38-d591-4992-b16b-68d2be4f756f None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:06.635756 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0914e38-d591-4992-b16b-68d2be4f756f None admin] PATH_INFO: `/v3/services` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:06.637457 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0914e38-d591-4992-b16b-68d2be4f756f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:06.637918 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0914e38-d591-4992-b16b-68d2be4f756f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:06.639482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0914e38-d591-4992-b16b-68d2be4f756f None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:06.646167 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 300/613] 10.222.0.30 () {56 vars in 1126 bytes} [Wed Nov 2 07:16:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:06.647217 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0914e38-d591-4992-b16b-68d2be4f756f None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:06.701812 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:06.714791 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672883b-ec98-41eb-a86a-f6ec5d486cbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:06.717006 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672883b-ec98-41eb-a86a-f6ec5d486cbf None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:06.754502 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0914e38-d591-4992-b16b-68d2be4f756f None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:06.789462 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a0fa9-db32-4fb2-b53d-7f8cf4295ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:06.790072 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a0fa9-db32-4fb2-b53d-7f8cf4295ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:06.790350 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a0fa9-db32-4fb2-b53d-7f8cf4295ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:06.790614 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a0fa9-db32-4fb2-b53d-7f8cf4295ed1 None admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:06.791491 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152a0fa9-db32-4fb2-b53d-7f8cf4295ed1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:06.792020 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152a0fa9-db32-4fb2-b53d-7f8cf4295ed1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:06.793494 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152a0fa9-db32-4fb2-b53d-7f8cf4295ed1 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:06.810222 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 301/614] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 07:16:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:06.810222 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc9f18-ef43-4bfa-bf91-e0d0e957af31 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:06.810222 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc9f18-ef43-4bfa-bf91-e0d0e957af31 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:06.810222 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc9f18-ef43-4bfa-bf91-e0d0e957af31 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:06.810222 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 302/615] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:06.810222 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc9f18-ef43-4bfa-bf91-e0d0e957af31 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:06.833827 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 314/616] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 07:16:06 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:16:06.833827 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672883b-ec98-41eb-a86a-f6ec5d486cbf None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:06.873412 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:06.929093 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:06.929864 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc9f18-ef43-4bfa-bf91-e0d0e957af31 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:06.936461 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672883b-ec98-41eb-a86a-f6ec5d486cbf None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:06.965919 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4424456-4354-4e2f-9605-3022d89e874e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:06.966536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4424456-4354-4e2f-9605-3022d89e874e None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:06.966845 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4424456-4354-4e2f-9605-3022d89e874e None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:06.967112 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4424456-4354-4e2f-9605-3022d89e874e None admin] PATH_INFO: `/v3/services` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:06.967746 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4424456-4354-4e2f-9605-3022d89e874e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:06.968049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4424456-4354-4e2f-9605-3022d89e874e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:06.969453 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4424456-4354-4e2f-9605-3022d89e874e None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:06.973138 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa4f37-109c-4dec-a131-63672142ee15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:06.974031 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa4f37-109c-4dec-a131-63672142ee15 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:06.974643 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 303/617] 10.222.0.30 () {56 vars in 1126 bytes} [Wed Nov 2 07:16:06 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:06.975235 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa4f37-109c-4dec-a131-63672142ee15 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:06.975666 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa4f37-109c-4dec-a131-63672142ee15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:06.976002 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398bdff-626b-420e-a72b-3805c5cd6e8f None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:06.976281 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398bdff-626b-420e-a72b-3805c5cd6e8f None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:06.976550 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398bdff-626b-420e-a72b-3805c5cd6e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:06.977036 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa4f37-109c-4dec-a131-63672142ee15 None admin] RBAC: Adding query filter params (role.id=1e35bb725eb8475ea63e2d9538ab67d5, scope.project.id=25e2bc8417754f909b416b93d5b7afb7, user.id=836c4044aaf74de391219289b57d2956) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:06.977477 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa4f37-109c-4dec-a131-63672142ee15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:06.979275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa4f37-109c-4dec-a131-63672142ee15 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:06.996876 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 315/618] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 07:16:06 2022] GET /identity/v3/role_assignments?user.id=836c4044aaf74de391219289b57d2956&scope.project.id=25e2bc8417754f909b416b93d5b7afb7&role.id=1e35bb725eb8475ea63e2d9538ab67d5 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:06.997615 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-4398bdff-626b-420e-a72b-3805c5cd6e8f None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:06.998412 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa4f37-109c-4dec-a131-63672142ee15 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:07.001711 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-4398bdff-626b-420e-a72b-3805c5cd6e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:07.122052 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 304/619] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:07.123949 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73cd422-2fdc-4a9d-b1ba-bd09f934f1f0 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:07.124395 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73cd422-2fdc-4a9d-b1ba-bd09f934f1f0 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:07.124837 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73cd422-2fdc-4a9d-b1ba-bd09f934f1f0 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:07.126685 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 305/620] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:07.136258 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797a377-4a08-4f86-a3b7-24e1f1614f77 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:07.136717 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797a377-4a08-4f86-a3b7-24e1f1614f77 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:07.136861 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797a377-4a08-4f86-a3b7-24e1f1614f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:07.176363 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-7797a377-4a08-4f86-a3b7-24e1f1614f77 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:07.177584 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-7797a377-4a08-4f86-a3b7-24e1f1614f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:07.180195 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa4f37-109c-4dec-a131-63672142ee15 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:07.226839 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc042b-b156-41d1-be86-b7d9133f3e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:07.228719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc042b-b156-41d1-be86-b7d9133f3e8a None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:07.228899 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc042b-b156-41d1-be86-b7d9133f3e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:07.229041 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc042b-b156-41d1-be86-b7d9133f3e8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:07.243916 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fc042b-b156-41d1-be86-b7d9133f3e8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:07.245904 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fc042b-b156-41d1-be86-b7d9133f3e8a None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:07.257690 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:07.265091 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-d0fc042b-b156-41d1-be86-b7d9133f3e8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:07.265091 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:07.265091 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:07.265091 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:07.265091 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:07.265091 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:07.265091 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:07.265091 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:07.265091 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:07.265091 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:07.265091 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:07.265091 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:07.265091 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:16:07.265091 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:16:07.265091 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:16:07.265091 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:16:07.266069 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:07.266069 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:07.266069 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:07.266069 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:07.266069 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:07.266069 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:07.266069 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:07.266069 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:07.266069 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:07.266069 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:07.266069 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:07.266069 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:07.266069 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:07.266069 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:07.266069 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:07.266069 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:07.266069 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:16:07.267128 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:16:07.267128 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:07.267128 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:07.267128 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 316/621] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:07.273738 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc042b-b156-41d1-be86-b7d9133f3e8a None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:07.292420 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 306/622] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:07.293945 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:07.402987 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc042b-b156-41d1-be86-b7d9133f3e8a None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:07.452711 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c220d4-4585-4f39-8c8e-d4b4fe480576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:07.453331 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c220d4-4585-4f39-8c8e-d4b4fe480576 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:07.453608 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c220d4-4585-4f39-8c8e-d4b4fe480576 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:07.453832 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7797a377-4a08-4f86-a3b7-24e1f1614f77 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:07.453942 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c220d4-4585-4f39-8c8e-d4b4fe480576 None admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:07.454509 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c220d4-4585-4f39-8c8e-d4b4fe480576 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:07.454833 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c220d4-4585-4f39-8c8e-d4b4fe480576 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:07.456369 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c220d4-4585-4f39-8c8e-d4b4fe480576 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:07.469308 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 317/623] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:16:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:07.475841 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c220d4-4585-4f39-8c8e-d4b4fe480576 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:07.590498 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7797a377-4a08-4f86-a3b7-24e1f1614f77 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:07.604399 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:07.631868 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0391ec-8bae-4eb7-bd57-2b62e1eb63e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:07.632992 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0391ec-8bae-4eb7-bd57-2b62e1eb63e5 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:07.633273 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0391ec-8bae-4eb7-bd57-2b62e1eb63e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:07.633540 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0391ec-8bae-4eb7-bd57-2b62e1eb63e5 None admin] PATH_INFO: `/v3/services/network` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:07.634149 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0391ec-8bae-4eb7-bd57-2b62e1eb63e5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:07.635974 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0391ec-8bae-4eb7-bd57-2b62e1eb63e5 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:07.643346 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c220d4-4585-4f39-8c8e-d4b4fe480576 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:07.647447 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-fb0391ec-8bae-4eb7-bd57-2b62e1eb63e5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:16:07.647447 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:07.647447 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:07.647447 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:07.647447 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:07.647447 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:07.647447 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:07.647447 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:07.647447 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:16:07.647447 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:07.647447 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:07.647447 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:07.647447 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:07.647447 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:07.647447 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:07.647447 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:07.647447 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:07.648412 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:07.648412 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:07.648412 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:07.648412 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:07.648412 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:07.648412 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:07.648412 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:07.648412 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:07.648412 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:07.648412 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:07.648412 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:07.648412 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:07.648412 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:07.648412 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:07.648412 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:07.648412 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:07.649247 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:07.649247 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:07.649247 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:07.649247 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:07.649247 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:07.649247 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:07.649247 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:07.649247 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:07.649247 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:07.649247 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:07.649247 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:07.649247 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:07.649247 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 07:16:07.649247 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:07.649247 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 307/624] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:16:07 2022] GET /identity/v3/services/network => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:07.650029 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0391ec-8bae-4eb7-bd57-2b62e1eb63e5 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:07.690228 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd293d-e931-4939-9400-fc306b3b872e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:07.690815 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd293d-e931-4939-9400-fc306b3b872e None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:07.691094 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd293d-e931-4939-9400-fc306b3b872e None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:07.691368 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd293d-e931-4939-9400-fc306b3b872e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:07.698374 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdd293d-e931-4939-9400-fc306b3b872e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:07.699878 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdd293d-e931-4939-9400-fc306b3b872e None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:07.703362 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:07.711526 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-dcdd293d-e931-4939-9400-fc306b3b872e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:16:07.711526 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:07.711526 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:07.711526 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:07.711526 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:07.711526 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:07.711526 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:07.711526 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:07.711526 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:07.711526 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:07.711526 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:07.711526 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:07.711526 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:16:07.711526 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:16:07.711526 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:16:07.712431 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:16:07.712431 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:07.712431 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:07.712431 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:07.712431 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:07.712431 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:07.712431 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:07.712431 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:07.712431 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:07.712431 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:07.712431 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:07.712431 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:07.712431 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:07.712431 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:07.712431 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:07.712431 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:07.712431 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:07.713428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:16:07.713428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:16:07.713428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:16:07.713428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:16:07.713428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:16:07.713428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:16:07.713428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:16:07.713428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:07.713428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 318/625] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:16:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:07.713980 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd293d-e931-4939-9400-fc306b3b872e None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:07.781900 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0391ec-8bae-4eb7-bd57-2b62e1eb63e5 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:07.825637 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e1165-1483-4709-9f49-1ff351c17c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:07.826703 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e1165-1483-4709-9f49-1ff351c17c3d None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:07.827614 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e1165-1483-4709-9f49-1ff351c17c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:07.828086 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e1165-1483-4709-9f49-1ff351c17c3d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:07.829085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680e1165-1483-4709-9f49-1ff351c17c3d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:07.831249 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680e1165-1483-4709-9f49-1ff351c17c3d None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:07.840377 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd293d-e931-4939-9400-fc306b3b872e None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:07.843677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-680e1165-1483-4709-9f49-1ff351c17c3d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:16:07.843677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:07.843677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:07.843677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:07.843677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:07.843677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:07.843677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:07.843677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:07.843677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:16:07.843677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:07.843677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:07.843677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:07.843677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:07.843677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:07.843677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:07.843677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:07.843677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:07.844629 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:07.844629 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:07.844629 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:07.844629 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:07.844629 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:07.844629 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:07.844629 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:07.844629 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:07.844629 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:07.844629 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:07.844629 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:07.844629 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:07.844629 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:07.844629 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:07.844629 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:07.844629 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:07.845431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:07.845431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:07.845431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:07.845431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:07.845431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:07.845431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:07.845431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:07.845431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:07.845431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:07.845431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:07.845431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:07.845431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:07.845431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 07:16:07.845431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:07.848557 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 308/626] 10.222.0.30 () {56 vars in 1129 bytes} [Wed Nov 2 07:16:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:07.849975 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e1165-1483-4709-9f49-1ff351c17c3d None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:07.887742 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f42fccb-919c-4744-ae7d-5cb3927bd85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:07.888362 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f42fccb-919c-4744-ae7d-5cb3927bd85d None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:07.888634 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f42fccb-919c-4744-ae7d-5cb3927bd85d None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:07.888901 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f42fccb-919c-4744-ae7d-5cb3927bd85d None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:07.889627 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f42fccb-919c-4744-ae7d-5cb3927bd85d None admin] RBAC: Adding query filter params (name=network) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:07.889934 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f42fccb-919c-4744-ae7d-5cb3927bd85d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:07.891513 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f42fccb-919c-4744-ae7d-5cb3927bd85d None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:07.896563 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 319/627] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Nov 2 07:16:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:07.897481 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f42fccb-919c-4744-ae7d-5cb3927bd85d None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:07.911026 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:07.970157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e1165-1483-4709-9f49-1ff351c17c3d None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:07.972403 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:08.008909 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc95b6b-cb21-4f8c-98e2-6b5bc5ecdb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:08.009932 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc95b6b-cb21-4f8c-98e2-6b5bc5ecdb10 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:08.010164 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc95b6b-cb21-4f8c-98e2-6b5bc5ecdb10 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.010497 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc95b6b-cb21-4f8c-98e2-6b5bc5ecdb10 None admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.011224 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc95b6b-cb21-4f8c-98e2-6b5bc5ecdb10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:08.012056 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc95b6b-cb21-4f8c-98e2-6b5bc5ecdb10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:08.013875 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc95b6b-cb21-4f8c-98e2-6b5bc5ecdb10 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:08.026290 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f42fccb-919c-4744-ae7d-5cb3927bd85d None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:08.033248 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 309/628] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 07:16:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:08.034292 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc95b6b-cb21-4f8c-98e2-6b5bc5ecdb10 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:08.071449 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742437ee-6c6b-46a6-8e71-0792c3208567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:08.072546 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742437ee-6c6b-46a6-8e71-0792c3208567 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:08.073029 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742437ee-6c6b-46a6-8e71-0792c3208567 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.073569 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742437ee-6c6b-46a6-8e71-0792c3208567 None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.074976 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742437ee-6c6b-46a6-8e71-0792c3208567 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:08.076205 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742437ee-6c6b-46a6-8e71-0792c3208567 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:08.078772 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742437ee-6c6b-46a6-8e71-0792c3208567 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:08.084718 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 320/629] 10.222.0.30 () {56 vars in 1124 bytes} [Wed Nov 2 07:16:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:08.085749 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742437ee-6c6b-46a6-8e71-0792c3208567 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:08.153340 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:08.162210 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc95b6b-cb21-4f8c-98e2-6b5bc5ecdb10 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:08.203760 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c4670-9e5f-4ca2-89ea-67b44605914d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:08.204372 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c4670-9e5f-4ca2-89ea-67b44605914d None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:08.204647 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c4670-9e5f-4ca2-89ea-67b44605914d None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.204978 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c4670-9e5f-4ca2-89ea-67b44605914d None admin] PATH_INFO: `/v3/services` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.205528 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c4670-9e5f-4ca2-89ea-67b44605914d None admin] RBAC: Adding query filter params (type=network) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:08.205884 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c4670-9e5f-4ca2-89ea-67b44605914d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:08.207289 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c4670-9e5f-4ca2-89ea-67b44605914d None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:08.207866 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742437ee-6c6b-46a6-8e71-0792c3208567 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:08.213648 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 310/630] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Nov 2 07:16:08 2022] GET /identity/v3/services?type=network => generated 350 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:08.214554 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c4670-9e5f-4ca2-89ea-67b44605914d None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:08.247220 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:08.250165 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c31b3-1846-4f7a-a66f-31b8896344a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:08.251129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c31b3-1846-4f7a-a66f-31b8896344a7 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:08.251560 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c31b3-1846-4f7a-a66f-31b8896344a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.251961 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c31b3-1846-4f7a-a66f-31b8896344a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.253381 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c31b3-1846-4f7a-a66f-31b8896344a7 None admin] RBAC: Adding query filter params (role.id=268fecadd98446f2b7d41755f3fe3649, scope.project.id=25e2bc8417754f909b416b93d5b7afb7, user.id=364555ec385c4dbb91907fb9c5116d2a) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:08.253844 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c31b3-1846-4f7a-a66f-31b8896344a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:08.255447 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c31b3-1846-4f7a-a66f-31b8896344a7 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:08.263376 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 321/631] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 07:16:08 2022] GET /identity/v3/role_assignments?user.id=364555ec385c4dbb91907fb9c5116d2a&scope.project.id=25e2bc8417754f909b416b93d5b7afb7&role.id=268fecadd98446f2b7d41755f3fe3649 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:08.264352 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c31b3-1846-4f7a-a66f-31b8896344a7 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:08.385707 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c4670-9e5f-4ca2-89ea-67b44605914d None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:08.436654 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c957e2-549d-42c4-b0f3-17abb21632a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:08.437678 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c957e2-549d-42c4-b0f3-17abb21632a1 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:08.438164 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c957e2-549d-42c4-b0f3-17abb21632a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.438617 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c957e2-549d-42c4-b0f3-17abb21632a1 None admin] PATH_INFO: `/v3/services` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.439467 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c957e2-549d-42c4-b0f3-17abb21632a1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:08.439977 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c957e2-549d-42c4-b0f3-17abb21632a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:08.441563 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c957e2-549d-42c4-b0f3-17abb21632a1 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:08.448969 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 311/632] 10.222.0.30 () {56 vars in 1124 bytes} [Wed Nov 2 07:16:08 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:08.457045 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c957e2-549d-42c4-b0f3-17abb21632a1 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:08.476952 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c31b3-1846-4f7a-a66f-31b8896344a7 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:08.523960 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:08.545018 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce07d450-fd60-4435-a8a3-5e5b1018ea08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:08.545636 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07d450-fd60-4435-a8a3-5e5b1018ea08 None admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:08.545916 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07d450-fd60-4435-a8a3-5e5b1018ea08 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.546185 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07d450-fd60-4435-a8a3-5e5b1018ea08 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.546890 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce07d450-fd60-4435-a8a3-5e5b1018ea08 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:08.548501 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce07d450-fd60-4435-a8a3-5e5b1018ea08 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:08.559581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:08.574065 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c957e2-549d-42c4-b0f3-17abb21632a1 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:08.586898 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 322/633] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:16:08 2022] POST /identity/v3/endpoints => generated 337 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:08.626426 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb474af-35a9-4866-97d0-1ea4d097ffe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:08.627098 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb474af-35a9-4866-97d0-1ea4d097ffe5 None admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:08.627417 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb474af-35a9-4866-97d0-1ea4d097ffe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:08.627682 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb474af-35a9-4866-97d0-1ea4d097ffe5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:08.628370 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb474af-35a9-4866-97d0-1ea4d097ffe5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:08.629773 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb474af-35a9-4866-97d0-1ea4d097ffe5 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:08.659191 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 312/634] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:16:08 2022] POST /identity/v3/endpoints => generated 346 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:09.352932 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5fbaf-0408-4a3f-94a6-f2b2225bd5e4 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:09.353497 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5fbaf-0408-4a3f-94a6-f2b2225bd5e4 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:09.353574 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5fbaf-0408-4a3f-94a6-f2b2225bd5e4 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:09.355266 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 323/635] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:09.360990 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f18cb5-81c1-42e9-bd43-26ce8bf6ecbe None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:09.361423 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f18cb5-81c1-42e9-bd43-26ce8bf6ecbe None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:09.361598 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f18cb5-81c1-42e9-bd43-26ce8bf6ecbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:09.382257 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-b0f18cb5-81c1-42e9-bd43-26ce8bf6ecbe None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:09.383895 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-b0f18cb5-81c1-42e9-bd43-26ce8bf6ecbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:09.470796 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 313/636] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:09.495680 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f63fe9-f73c-4938-8aaf-b4cd7864e829 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:09.495978 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f63fe9-f73c-4938-8aaf-b4cd7864e829 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:09.496225 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f63fe9-f73c-4938-8aaf-b4cd7864e829 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:09.497806 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 324/637] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:09.503105 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc98c4e-d8de-4184-acbf-9f70a34a1b61 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:09.503391 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc98c4e-d8de-4184-acbf-9f70a34a1b61 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:09.503711 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc98c4e-d8de-4184-acbf-9f70a34a1b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:09.523559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-6dc98c4e-d8de-4184-acbf-9f70a34a1b61 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:09.525103 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-6dc98c4e-d8de-4184-acbf-9f70a34a1b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:09.593830 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:09.598725 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 314/638] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:09.740034 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f63fe9-f73c-4938-8aaf-b4cd7864e829 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:09.846816 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f63fe9-f73c-4938-8aaf-b4cd7864e829 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:09.859979 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc98c4e-d8de-4184-acbf-9f70a34a1b61 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:09.884242 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb0da6-3647-4170-8641-4091084ff3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:09.885229 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb0da6-3647-4170-8641-4091084ff3b3 None admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:09.885697 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb0da6-3647-4170-8641-4091084ff3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:09.886167 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb0da6-3647-4170-8641-4091084ff3b3 None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:09.887063 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb0da6-3647-4170-8641-4091084ff3b3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:09.888627 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb0da6-3647-4170-8641-4091084ff3b3 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:09.908469 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 325/639] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Nov 2 07:16:09 2022] POST /identity/v3/services => generated 269 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:09.975866 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc98c4e-d8de-4184-acbf-9f70a34a1b61 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:10.024140 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d618be-da10-4ac2-b46c-82b643d5198f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:10.024927 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d618be-da10-4ac2-b46c-82b643d5198f None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:10.025228 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d618be-da10-4ac2-b46c-82b643d5198f None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:10.025561 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d618be-da10-4ac2-b46c-82b643d5198f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:10.026203 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d618be-da10-4ac2-b46c-82b643d5198f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:10.028023 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d618be-da10-4ac2-b46c-82b643d5198f None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:10.038013 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-81d618be-da10-4ac2-b46c-82b643d5198f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:16:10.038013 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:10.038013 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:10.038013 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:10.038013 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:10.038013 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:10.038013 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:10.038013 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:10.038013 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:16:10.038013 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:10.038013 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:10.038013 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:10.038013 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:10.038013 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:10.038013 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:10.038013 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:10.038013 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:10.039061 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:10.039061 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:10.039061 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:10.039061 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:10.039061 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:10.039061 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:10.039061 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:10.039061 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:10.039061 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:10.039061 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:10.039061 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:10.039061 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:10.039061 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:10.039061 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:10.039061 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:10.039061 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:10.039885 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:10.039885 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:10.039885 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:10.039885 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:10.039885 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:10.039885 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:10.039885 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:10.039885 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:10.039885 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:10.039885 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:10.039885 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:10.039885 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:10.039885 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:16:10.039885 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:10.039885 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 315/640] 10.222.0.30 () {56 vars in 1126 bytes} [Wed Nov 2 07:16:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:10.047141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb0da6-3647-4170-8641-4091084ff3b3 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:10.102401 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:10.171590 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb0da6-3647-4170-8641-4091084ff3b3 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:10.212260 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49c2f28-48f0-471f-b503-0f396213bd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:10.213228 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c2f28-48f0-471f-b503-0f396213bd37 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:10.213717 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c2f28-48f0-471f-b503-0f396213bd37 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:10.214176 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c2f28-48f0-471f-b503-0f396213bd37 None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:10.215022 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49c2f28-48f0-471f-b503-0f396213bd37 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:10.215549 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49c2f28-48f0-471f-b503-0f396213bd37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:10.217135 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49c2f28-48f0-471f-b503-0f396213bd37 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:10.222646 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 326/641] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Nov 2 07:16:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:10.229060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d618be-da10-4ac2-b46c-82b643d5198f None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:10.295066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:10.333164 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d618be-da10-4ac2-b46c-82b643d5198f None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:10.367887 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3cecee-9ae2-42ae-938d-29f3515c3a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:10.369429 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3cecee-9ae2-42ae-938d-29f3515c3a36 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:10.369933 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3cecee-9ae2-42ae-938d-29f3515c3a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:10.370419 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3cecee-9ae2-42ae-938d-29f3515c3a36 None admin] PATH_INFO: `/v3/services` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:10.371279 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3cecee-9ae2-42ae-938d-29f3515c3a36 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:10.371781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3cecee-9ae2-42ae-938d-29f3515c3a36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:10.373391 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3cecee-9ae2-42ae-938d-29f3515c3a36 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:10.378571 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 316/642] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Nov 2 07:16:10 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:10.850699 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b0e2e-025f-4ae0-a27a-4ad9d7ab28e3 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:10.850699 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b0e2e-025f-4ae0-a27a-4ad9d7ab28e3 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:10.850699 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b0e2e-025f-4ae0-a27a-4ad9d7ab28e3 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:10.852367 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 327/643] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:10.858240 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98de15-904c-4937-8222-d5012fd1f569 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:10.858814 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98de15-904c-4937-8222-d5012fd1f569 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:10.859280 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98de15-904c-4937-8222-d5012fd1f569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:10.878285 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-de98de15-904c-4937-8222-d5012fd1f569 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:10.879842 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-de98de15-904c-4937-8222-d5012fd1f569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:10.950003 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 317/644] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:10 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:16:11.076231 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8cd642-39ed-4c41-92bd-d7e84755edb2 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:11.076645 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8cd642-39ed-4c41-92bd-d7e84755edb2 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:11.076747 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8cd642-39ed-4c41-92bd-d7e84755edb2 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:11.078402 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 328/645] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:11.088795 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a940e05-e765-4435-81f4-59b84251c244 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:11.089061 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a940e05-e765-4435-81f4-59b84251c244 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:11.089327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a940e05-e765-4435-81f4-59b84251c244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:11.109723 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-1a940e05-e765-4435-81f4-59b84251c244 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:11.110866 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-1a940e05-e765-4435-81f4-59b84251c244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:11.162077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:11.183002 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 318/646] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:11 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:11.237566 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8cd642-39ed-4c41-92bd-d7e84755edb2 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:11.346230 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8cd642-39ed-4c41-92bd-d7e84755edb2 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:11.348595 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:11.381514 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22466ec8-9e5d-4107-813d-07a2cbd4f748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:11.382070 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22466ec8-9e5d-4107-813d-07a2cbd4f748 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:11.382344 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22466ec8-9e5d-4107-813d-07a2cbd4f748 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:11.382616 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22466ec8-9e5d-4107-813d-07a2cbd4f748 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:11.393331 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22466ec8-9e5d-4107-813d-07a2cbd4f748 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:11.398141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-22466ec8-9e5d-4107-813d-07a2cbd4f748 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 07:16:11.398141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:11.398141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:11.398141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:11.398141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:11.398141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:11.398141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:11.398141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:11.398141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:11.398141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:11.398141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:11.398141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:11.398141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:11.398141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:11.398141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:11.398141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:11.399082 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:11.399082 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:11.399082 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:11.399082 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:11.399082 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:11.399082 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:11.399082 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:11.399082 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:11.399082 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:11.399082 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:11.399082 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:11.399082 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:11.399082 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:11.399082 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:11.399082 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:11.399082 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:11.400010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:11.400010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:11.400010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:11.400010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:11.400010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:11.400010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:11.400010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:11.400010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:11.400010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:11.400010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:11.400010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 07:16:11.400010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:11.400010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 329/647] 10.222.0.30 () {56 vars in 1120 bytes} [Wed Nov 2 07:16:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:11.406124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a940e05-e765-4435-81f4-59b84251c244 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:11.481292 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22466ec8-9e5d-4107-813d-07a2cbd4f748 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:11.512630 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a940e05-e765-4435-81f4-59b84251c244 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:11.548963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a007c-2730-4f42-ab17-43237f40c95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:11.549814 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a007c-2730-4f42-ab17-43237f40c95b None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:11.550225 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a007c-2730-4f42-ab17-43237f40c95b None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:11.550625 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a007c-2730-4f42-ab17-43237f40c95b None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:11.551461 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a007c-2730-4f42-ab17-43237f40c95b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:11.551892 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a007c-2730-4f42-ab17-43237f40c95b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:11.554127 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a007c-2730-4f42-ab17-43237f40c95b None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:11.563009 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 319/648] 10.222.0.30 () {56 vars in 1115 bytes} [Wed Nov 2 07:16:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:11.571837 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a007c-2730-4f42-ab17-43237f40c95b None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:11.598488 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22466ec8-9e5d-4107-813d-07a2cbd4f748 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:11.624561 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:11.637640 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0f6d9b-ef07-41db-a167-81d92c2896d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:11.638591 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f6d9b-ef07-41db-a167-81d92c2896d3 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:11.639018 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f6d9b-ef07-41db-a167-81d92c2896d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:11.639533 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f6d9b-ef07-41db-a167-81d92c2896d3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:11.640309 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0f6d9b-ef07-41db-a167-81d92c2896d3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:11.641693 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0f6d9b-ef07-41db-a167-81d92c2896d3 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:11.653659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-8c0f6d9b-ef07-41db-a167-81d92c2896d3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:16:11.653659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:11.653659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:11.653659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:11.653659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:11.653659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:11.653659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:11.653659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:11.653659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:16:11.653659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:11.653659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:11.653659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:11.653659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:11.653659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:11.653659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:11.653659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:11.653659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:11.654695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:11.654695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:11.654695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:11.654695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:11.654695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:11.654695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:11.654695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:11.654695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:11.654695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:11.654695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:11.654695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:11.654695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:11.654695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:11.654695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:11.654695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:11.654695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:11.660640 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:11.660640 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:11.660640 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:11.660640 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:11.660640 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:11.660640 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:11.660640 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:11.660640 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:11.660640 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:11.660640 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:11.660640 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:11.660640 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:11.660640 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:16:11.660640 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:11.660640 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 330/649] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Nov 2 07:16:11 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:11.663027 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0f6d9b-ef07-41db-a167-81d92c2896d3 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:11.688476 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a007c-2730-4f42-ab17-43237f40c95b None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:11.701263 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:11.724795 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd11980-b43e-4951-b223-99c9c39bbb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:11.725740 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd11980-b43e-4951-b223-99c9c39bbb23 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:11.726212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd11980-b43e-4951-b223-99c9c39bbb23 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:11.726478 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd11980-b43e-4951-b223-99c9c39bbb23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:11.737836 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd11980-b43e-4951-b223-99c9c39bbb23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:11.739315 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd11980-b43e-4951-b223-99c9c39bbb23 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:11.753919 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-2fd11980-b43e-4951-b223-99c9c39bbb23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:11.753919 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:11.753919 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:11.753919 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:11.753919 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:11.753919 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:11.753919 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:11.753919 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:11.753919 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:11.753919 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:11.753919 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:11.753919 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:11.753919 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:16:11.753919 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:16:11.753919 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:16:11.753919 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:16:11.754926 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:11.754926 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:11.754926 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:11.754926 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:11.754926 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:11.754926 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:11.754926 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:11.754926 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:11.754926 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:11.754926 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:11.754926 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:11.754926 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:11.754926 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:11.754926 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:11.754926 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:11.754926 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:11.754926 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:16:11.755831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:16:11.755831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:11.755831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:11.759097 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 320/650] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:11.761212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd11980-b43e-4951-b223-99c9c39bbb23 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:11.795304 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0f6d9b-ef07-41db-a167-81d92c2896d3 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:11.830606 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479fcb4-3a72-49c4-9904-2d3916ca4f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:11.831286 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479fcb4-3a72-49c4-9904-2d3916ca4f7a None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:11.831560 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479fcb4-3a72-49c4-9904-2d3916ca4f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:11.831822 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479fcb4-3a72-49c4-9904-2d3916ca4f7a None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:11.832538 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e479fcb4-3a72-49c4-9904-2d3916ca4f7a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:11.832834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e479fcb4-3a72-49c4-9904-2d3916ca4f7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:11.834295 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e479fcb4-3a72-49c4-9904-2d3916ca4f7a None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:11.841702 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 331/651] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Nov 2 07:16:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:11.847945 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479fcb4-3a72-49c4-9904-2d3916ca4f7a None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:11.865503 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:11.869544 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd11980-b43e-4951-b223-99c9c39bbb23 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:11.904414 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f30ed1-1c4d-4b44-9a56-827ab58f5697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:11.904967 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f30ed1-1c4d-4b44-9a56-827ab58f5697 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:11.905240 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f30ed1-1c4d-4b44-9a56-827ab58f5697 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:11.905508 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f30ed1-1c4d-4b44-9a56-827ab58f5697 None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:11.906140 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f30ed1-1c4d-4b44-9a56-827ab58f5697 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:11.906441 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f30ed1-1c4d-4b44-9a56-827ab58f5697 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:11.907859 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f30ed1-1c4d-4b44-9a56-827ab58f5697 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:11.925465 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 321/652] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:16:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:11.926378 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f30ed1-1c4d-4b44-9a56-827ab58f5697 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:11.951166 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:11.959221 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479fcb4-3a72-49c4-9904-2d3916ca4f7a None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:11.995937 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6300b38-9220-4482-a881-538962c8b7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:11.996714 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6300b38-9220-4482-a881-538962c8b7c3 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:11.997079 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6300b38-9220-4482-a881-538962c8b7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:11.997489 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6300b38-9220-4482-a881-538962c8b7c3 None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:11.998321 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6300b38-9220-4482-a881-538962c8b7c3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:11.998758 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6300b38-9220-4482-a881-538962c8b7c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:12.000226 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6300b38-9220-4482-a881-538962c8b7c3 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:12.011040 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 332/653] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Nov 2 07:16:11 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:12.032567 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f30ed1-1c4d-4b44-9a56-827ab58f5697 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:12.070510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208bc7c-fccc-4b4d-af11-d9ee50dbe6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:12.071199 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208bc7c-fccc-4b4d-af11-d9ee50dbe6b8 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:12.071509 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208bc7c-fccc-4b4d-af11-d9ee50dbe6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:12.071823 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208bc7c-fccc-4b4d-af11-d9ee50dbe6b8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:12.080469 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6208bc7c-fccc-4b4d-af11-d9ee50dbe6b8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:12.081972 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6208bc7c-fccc-4b4d-af11-d9ee50dbe6b8 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:12.097057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-6208bc7c-fccc-4b4d-af11-d9ee50dbe6b8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 07:16:12.097057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:12.097057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:12.097057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:12.097057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:12.097057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:12.097057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:12.097057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:12.097057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:12.097057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:12.097057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:12.097057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:12.097057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:16:12.097057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:16:12.097057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:16:12.097057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:16:12.098482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:12.098482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:12.098482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:16:12.098482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:16:12.098482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:16:12.098482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:16:12.098482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:12.098482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:12.098482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:12.098482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:12.098482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:12.098482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:12.098482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:12.098482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:12.098482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:12.098482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:12.098482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:12.098482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:12.100441 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:12.100441 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:12.100441 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:16:12.100441 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:16:12.100441 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:16:12.100441 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:16:12.100441 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:16:12.100441 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:16:12.100441 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 07:16:12.100441 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:12.106715 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 322/654] 10.222.0.30 () {56 vars in 1117 bytes} [Wed Nov 2 07:16:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:12.106981 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6300b38-9220-4482-a881-538962c8b7c3 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:12.217381 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6300b38-9220-4482-a881-538962c8b7c3 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:12.256211 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a34a5-b369-410c-bf2c-44f8bf66fad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:12.256843 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a34a5-b369-410c-bf2c-44f8bf66fad4 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:12.257117 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a34a5-b369-410c-bf2c-44f8bf66fad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:12.257387 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a34a5-b369-410c-bf2c-44f8bf66fad4 None admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:12.258140 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4a34a5-b369-410c-bf2c-44f8bf66fad4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:12.258442 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4a34a5-b369-410c-bf2c-44f8bf66fad4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:12.260041 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4a34a5-b369-410c-bf2c-44f8bf66fad4 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:12.283733 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 333/655] 10.222.0.30 () {56 vars in 1149 bytes} [Wed Nov 2 07:16:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:12.287571 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208bc7c-fccc-4b4d-af11-d9ee50dbe6b8 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:12.308179 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:12.378974 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e423f1d-1818-4b49-9428-d45a4b20d989 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:12.379722 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e423f1d-1818-4b49-9428-d45a4b20d989 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:12.380225 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e423f1d-1818-4b49-9428-d45a4b20d989 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:12.382041 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 334/656] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:16:12.387188 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b2614-fcc3-4a4e-a424-f1355f1f4cff None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:12.387471 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b2614-fcc3-4a4e-a424-f1355f1f4cff None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:12.387745 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b2614-fcc3-4a4e-a424-f1355f1f4cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:12.392429 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208bc7c-fccc-4b4d-af11-d9ee50dbe6b8 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:12.406444 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-357b2614-fcc3-4a4e-a424-f1355f1f4cff None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:12.407998 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-357b2614-fcc3-4a4e-a424-f1355f1f4cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:12.420058 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:12.428789 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777361e-58e2-4630-85f4-8e1dfac657c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:12.429709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777361e-58e2-4630-85f4-8e1dfac657c5 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:12.429988 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777361e-58e2-4630-85f4-8e1dfac657c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:12.430253 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777361e-58e2-4630-85f4-8e1dfac657c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:12.441161 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8777361e-58e2-4630-85f4-8e1dfac657c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:12.442633 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8777361e-58e2-4630-85f4-8e1dfac657c5 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:12.461154 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-8777361e-58e2-4630-85f4-8e1dfac657c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:12.461154 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:12.461154 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:12.461154 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:12.461154 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:12.461154 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:12.461154 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:12.461154 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:12.461154 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:12.461154 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:12.461154 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:12.461154 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:12.461154 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:16:12.461154 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:16:12.461154 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:16:12.461154 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:16:12.462171 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:12.462171 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:12.462171 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:12.462171 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:12.462171 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:12.462171 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:12.462171 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:12.462171 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:12.462171 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:12.462171 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:12.462171 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:12.462171 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:12.462171 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:12.462171 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:12.462171 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:12.462171 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:12.462171 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:16:12.463710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:16:12.463710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:12.463710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:12.465745 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 323/657] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:12.471914 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777361e-58e2-4630-85f4-8e1dfac657c5 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:12.494683 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 335/658] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:12 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:16:12.557744 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:12.584940 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777361e-58e2-4630-85f4-8e1dfac657c5 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:12.619699 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a7b8c-36a1-4bcd-87bd-30e149a301d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:12.620453 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a7b8c-36a1-4bcd-87bd-30e149a301d5 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:12.620728 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a7b8c-36a1-4bcd-87bd-30e149a301d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:12.620992 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a7b8c-36a1-4bcd-87bd-30e149a301d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:12.621628 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a7b8c-36a1-4bcd-87bd-30e149a301d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:12.621925 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a7b8c-36a1-4bcd-87bd-30e149a301d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:12.623327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a7b8c-36a1-4bcd-87bd-30e149a301d5 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:12.636372 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 324/659] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:16:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:12.644114 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b2614-fcc3-4a4e-a424-f1355f1f4cff None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:12.671399 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f7e89-62d2-481f-8b23-95d7a4c2d711 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:12.671709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f7e89-62d2-481f-8b23-95d7a4c2d711 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:12.671944 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f7e89-62d2-481f-8b23-95d7a4c2d711 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:12.673523 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 325/660] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:16:12.678346 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e909c4-f0ac-4254-a662-83d7cdb370a6 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:12.678628 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e909c4-f0ac-4254-a662-83d7cdb370a6 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:12.678930 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e909c4-f0ac-4254-a662-83d7cdb370a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:12.701078 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-d2e909c4-f0ac-4254-a662-83d7cdb370a6 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:12.702379 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-d2e909c4-f0ac-4254-a662-83d7cdb370a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:12.756039 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b2614-fcc3-4a4e-a424-f1355f1f4cff None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:12.775709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 326/661] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:12 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:16:12.776975 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e909c4-f0ac-4254-a662-83d7cdb370a6 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:12.803506 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36817a36-8f38-4a0f-a425-b8653f08700d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:12.804796 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36817a36-8f38-4a0f-a425-b8653f08700d None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:12.805732 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36817a36-8f38-4a0f-a425-b8653f08700d None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:12.806503 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36817a36-8f38-4a0f-a425-b8653f08700d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:12.815920 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36817a36-8f38-4a0f-a425-b8653f08700d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:12.818152 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36817a36-8f38-4a0f-a425-b8653f08700d None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:12.833544 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-36817a36-8f38-4a0f-a425-b8653f08700d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:16:12.833544 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:12.833544 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:12.833544 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:12.833544 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:12.833544 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:12.833544 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:12.833544 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:12.833544 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:12.833544 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:12.833544 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:12.833544 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:12.833544 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:16:12.833544 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:16:12.833544 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:16:12.834484 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:16:12.834484 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:12.834484 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:12.834484 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:12.834484 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:12.834484 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:12.834484 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:12.834484 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:12.834484 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:12.834484 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:12.834484 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:12.834484 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:12.834484 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:12.834484 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:12.834484 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:12.834484 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:12.834484 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:12.835441 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:16:12.835441 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:16:12.835441 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:16:12.835441 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:16:12.835441 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:16:12.835441 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:16:12.835441 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:16:12.835441 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:12.841148 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 336/662] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:16:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:12.842360 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36817a36-8f38-4a0f-a425-b8653f08700d None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:12.846518 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:12.848716 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:12.898685 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e909c4-f0ac-4254-a662-83d7cdb370a6 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:12.935157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5e99f-54ae-4da5-a702-866d6dc0e247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:12.935840 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5e99f-54ae-4da5-a702-866d6dc0e247 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:12.936155 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5e99f-54ae-4da5-a702-866d6dc0e247 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:12.936422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5e99f-54ae-4da5-a702-866d6dc0e247 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:12.936999 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5e99f-54ae-4da5-a702-866d6dc0e247 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:12.938375 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5e99f-54ae-4da5-a702-866d6dc0e247 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:12.948042 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-dab5e99f-54ae-4da5-a702-866d6dc0e247 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:16:12.948042 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:12.948042 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:12.948042 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:12.948042 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:12.948042 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:12.948042 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:12.948042 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:12.948042 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:16:12.948042 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:12.948042 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:12.948042 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:12.948042 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:12.948042 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:12.948042 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:12.948042 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:12.948981 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:12.948981 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:12.948981 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:12.948981 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:12.948981 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:12.948981 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:12.948981 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:12.948981 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:12.948981 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:12.948981 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:12.948981 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:12.948981 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:12.948981 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:12.948981 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:12.948981 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:12.948981 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:12.948981 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:12.949853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:12.949853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:12.949853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:12.949853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:12.949853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:12.949853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:12.949853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:12.949853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:12.949853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:12.949853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:12.949853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:12.949853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:12.949853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:16:12.949853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:12.949853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 327/663] 10.222.0.30 () {56 vars in 1132 bytes} [Wed Nov 2 07:16:12 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:12.954497 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5e99f-54ae-4da5-a702-866d6dc0e247 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:12.981900 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36817a36-8f38-4a0f-a425-b8653f08700d None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:13.016802 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339d66f-4ec7-47cc-99ee-cc7936df19a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:13.017462 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339d66f-4ec7-47cc-99ee-cc7936df19a1 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:13.017638 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339d66f-4ec7-47cc-99ee-cc7936df19a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:13.017880 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339d66f-4ec7-47cc-99ee-cc7936df19a1 None admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:13.018678 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339d66f-4ec7-47cc-99ee-cc7936df19a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:13.019017 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339d66f-4ec7-47cc-99ee-cc7936df19a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:13.020763 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339d66f-4ec7-47cc-99ee-cc7936df19a1 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:13.041114 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 337/664] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 07:16:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:13.042556 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339d66f-4ec7-47cc-99ee-cc7936df19a1 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:13.081340 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5e99f-54ae-4da5-a702-866d6dc0e247 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:13.105946 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:13.107045 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:13.119124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d54cf06-0b68-47a1-a84f-5822f8233d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:13.119818 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54cf06-0b68-47a1-a84f-5822f8233d5d None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:13.120096 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54cf06-0b68-47a1-a84f-5822f8233d5d None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:13.120364 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54cf06-0b68-47a1-a84f-5822f8233d5d None admin] PATH_INFO: `/v3/services` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:13.120984 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d54cf06-0b68-47a1-a84f-5822f8233d5d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:13.121282 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d54cf06-0b68-47a1-a84f-5822f8233d5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:13.123150 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d54cf06-0b68-47a1-a84f-5822f8233d5d None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:13.128408 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 328/665] 10.222.0.30 () {56 vars in 1126 bytes} [Wed Nov 2 07:16:12 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:13.129158 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d54cf06-0b68-47a1-a84f-5822f8233d5d None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:13.151390 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339d66f-4ec7-47cc-99ee-cc7936df19a1 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:13.190177 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aa8a4c-bbf9-4c73-9fca-916bcafb4384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:13.190869 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa8a4c-bbf9-4c73-9fca-916bcafb4384 None admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:13.191141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa8a4c-bbf9-4c73-9fca-916bcafb4384 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:13.191398 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa8a4c-bbf9-4c73-9fca-916bcafb4384 None admin] PATH_INFO: `/v3/projects/25e2bc8417754f909b416b93d5b7afb7/users/364555ec385c4dbb91907fb9c5116d2a/roles/e6483a4fca514243bed809857f555411` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:13.193540 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aa8a4c-bbf9-4c73-9fca-916bcafb4384 None admin] RBAC: Authorizing `identity:create_grant(project_id=25e2bc8417754f909b416b93d5b7afb7, user_id=364555ec385c4dbb91907fb9c5116d2a, role_id=e6483a4fca514243bed809857f555411)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:13.195020 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aa8a4c-bbf9-4c73-9fca-916bcafb4384 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:13.211024 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 338/666] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 07:16:13 2022] PUT /identity/v3/projects/25e2bc8417754f909b416b93d5b7afb7/users/364555ec385c4dbb91907fb9c5116d2a/roles/e6483a4fca514243bed809857f555411 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:16:13.211970 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aa8a4c-bbf9-4c73-9fca-916bcafb4384 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:13.244088 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d54cf06-0b68-47a1-a84f-5822f8233d5d None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:13.296013 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed68cd-b3a2-411c-bd22-2857adee70f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:13.296982 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed68cd-b3a2-411c-bd22-2857adee70f7 None admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:13.297688 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed68cd-b3a2-411c-bd22-2857adee70f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:13.298119 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed68cd-b3a2-411c-bd22-2857adee70f7 None admin] PATH_INFO: `/v3/services` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:13.299748 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ed68cd-b3a2-411c-bd22-2857adee70f7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:13.302110 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ed68cd-b3a2-411c-bd22-2857adee70f7 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:13.326074 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 329/667] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Nov 2 07:16:13 2022] POST /identity/v3/services => generated 241 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:13.361186 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aa8a4c-bbf9-4c73-9fca-916bcafb4384 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:13.391913 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:13.412245 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c34a13-715c-42f0-9344-3d61556e08d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:13.412875 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c34a13-715c-42f0-9344-3d61556e08d3 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:13.413153 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c34a13-715c-42f0-9344-3d61556e08d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:13.413423 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c34a13-715c-42f0-9344-3d61556e08d3 None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:13.414059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c34a13-715c-42f0-9344-3d61556e08d3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:13.414360 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c34a13-715c-42f0-9344-3d61556e08d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:13.416294 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c34a13-715c-42f0-9344-3d61556e08d3 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:13.422795 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 339/668] 10.222.0.30 () {56 vars in 1126 bytes} [Wed Nov 2 07:16:13 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:13.429165 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed68cd-b3a2-411c-bd22-2857adee70f7 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:13.489121 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:13.548649 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed68cd-b3a2-411c-bd22-2857adee70f7 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:13.583034 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29841dbd-220c-4710-81e0-fb8959850fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:13.583649 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29841dbd-220c-4710-81e0-fb8959850fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:13.583922 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29841dbd-220c-4710-81e0-fb8959850fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:13.584189 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29841dbd-220c-4710-81e0-fb8959850fc2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:13.584882 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29841dbd-220c-4710-81e0-fb8959850fc2 None admin] RBAC: Adding query filter params (interface=public, service_id=711d6a4e4edf4b62acc9efb089743ac8, region_id=RegionOne) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:13.585183 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29841dbd-220c-4710-81e0-fb8959850fc2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:13.586639 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29841dbd-220c-4710-81e0-fb8959850fc2 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:13.595336 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 330/669] 10.222.0.30 () {56 vars in 1251 bytes} [Wed Nov 2 07:16:13 2022] GET /identity/v3/endpoints?service_id=711d6a4e4edf4b62acc9efb089743ac8&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:13.601127 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c34a13-715c-42f0-9344-3d61556e08d3 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:13.710052 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c34a13-715c-42f0-9344-3d61556e08d3 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:13.745641 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9457071f-1a94-4521-9429-628547c4bf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:13.746263 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9457071f-1a94-4521-9429-628547c4bf14 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:13.746536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9457071f-1a94-4521-9429-628547c4bf14 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:13.746836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9457071f-1a94-4521-9429-628547c4bf14 None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:13.747726 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9457071f-1a94-4521-9429-628547c4bf14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:13.749100 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9457071f-1a94-4521-9429-628547c4bf14 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:13.755906 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 340/670] 10.222.0.30 () {56 vars in 1087 bytes} [Wed Nov 2 07:16:13 2022] GET /identity/v3/services => generated 1258 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:16:14.329716 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1e27e-c99e-412e-ade2-a887d83db6f7 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:14.330157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1e27e-c99e-412e-ade2-a887d83db6f7 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:14.330260 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1e27e-c99e-412e-ade2-a887d83db6f7 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:14.331864 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 331/671] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:14.337317 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67ed47-77db-4dbd-a745-26a3e36a8923 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:14.337876 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67ed47-77db-4dbd-a745-26a3e36a8923 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:14.337876 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67ed47-77db-4dbd-a745-26a3e36a8923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:14.355974 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-8d67ed47-77db-4dbd-a745-26a3e36a8923 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:14.357156 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-8d67ed47-77db-4dbd-a745-26a3e36a8923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:14.404459 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:14.429980 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 341/672] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:14 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:14.707115 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1e27e-c99e-412e-ade2-a887d83db6f7 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:14.810691 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1e27e-c99e-412e-ade2-a887d83db6f7 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:14.823716 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:14.850318 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb8eb56-93ef-4378-8052-c62bcda5f3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:14.850958 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8eb56-93ef-4378-8052-c62bcda5f3a9 None admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:14.851235 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8eb56-93ef-4378-8052-c62bcda5f3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:14.851503 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8eb56-93ef-4378-8052-c62bcda5f3a9 None admin] PATH_INFO: `/v3/services` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:14.852223 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb8eb56-93ef-4378-8052-c62bcda5f3a9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:14.853724 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb8eb56-93ef-4378-8052-c62bcda5f3a9 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:14.871207 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 332/673] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Nov 2 07:16:14 2022] POST /identity/v3/services => generated 251 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:15.464203 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0617e-8dfc-45e9-bf93-31e9f2e9c637 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:15.464792 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0617e-8dfc-45e9-bf93-31e9f2e9c637 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:15.464792 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0617e-8dfc-45e9-bf93-31e9f2e9c637 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:15.466481 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 342/674] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:15.472454 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942b606-7544-4e8c-878f-96e35e8f21fd None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:15.472768 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942b606-7544-4e8c-878f-96e35e8f21fd None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:15.473063 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942b606-7544-4e8c-878f-96e35e8f21fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:15.492212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-c942b606-7544-4e8c-878f-96e35e8f21fd None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:15.493435 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-c942b606-7544-4e8c-878f-96e35e8f21fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:15.567117 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 333/675] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:15.572128 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07315cb3-44eb-44d2-82af-669de4ef243d None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:15.572665 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07315cb3-44eb-44d2-82af-669de4ef243d None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:15.573135 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07315cb3-44eb-44d2-82af-669de4ef243d None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:15.574921 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 343/676] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:15.579920 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3549bf-578c-49a6-8797-c1bda9334293 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:15.580204 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3549bf-578c-49a6-8797-c1bda9334293 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:15.580475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3549bf-578c-49a6-8797-c1bda9334293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:15.599215 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-8f3549bf-578c-49a6-8797-c1bda9334293 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:15.600412 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-8f3549bf-578c-49a6-8797-c1bda9334293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:15.669385 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 334/677] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:15.843682 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07315cb3-44eb-44d2-82af-669de4ef243d None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:15.935247 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3549bf-578c-49a6-8797-c1bda9334293 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:15.966456 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07315cb3-44eb-44d2-82af-669de4ef243d None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:15.978994 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:16.009409 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0cba3b-cdff-4d67-bebd-4e979611051d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:16.010334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0cba3b-cdff-4d67-bebd-4e979611051d None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:16.010807 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0cba3b-cdff-4d67-bebd-4e979611051d None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:16.011207 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0cba3b-cdff-4d67-bebd-4e979611051d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:16.021880 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0cba3b-cdff-4d67-bebd-4e979611051d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:16.027324 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-ca0cba3b-cdff-4d67-bebd-4e979611051d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 07:16:16.027324 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:16.027324 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:16.027324 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:16.027324 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:16.027324 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:16.027324 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:16.027324 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:16.027324 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:16.027324 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:16.027324 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:16.027324 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:16.027324 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:16.027324 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:16.027324 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:16.027324 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:16.028439 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:16.028439 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:16.028439 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:16.028439 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:16.028439 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:16.028439 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:16.028439 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:16.028439 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:16.028439 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:16.028439 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:16.028439 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:16.028439 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:16.028439 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:16.028439 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:16.028439 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:16.028439 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:16.029467 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:16.029467 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:16.029467 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:16.029467 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:16.029467 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:16.029467 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:16.029467 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:16.029467 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:16.029467 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:16.029467 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:16.029467 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 07:16:16.029467 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:16.031755 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 344/678] 10.222.0.30 () {56 vars in 1120 bytes} [Wed Nov 2 07:16:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:16.037953 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0cba3b-cdff-4d67-bebd-4e979611051d None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:16.074717 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3549bf-578c-49a6-8797-c1bda9334293 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:16.081733 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:16.114104 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13472d-c25a-4088-af11-4a5caebf890c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:16.115044 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13472d-c25a-4088-af11-4a5caebf890c None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:16.115320 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13472d-c25a-4088-af11-4a5caebf890c None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:16.115589 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13472d-c25a-4088-af11-4a5caebf890c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:16.116174 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a13472d-c25a-4088-af11-4a5caebf890c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:16.117774 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a13472d-c25a-4088-af11-4a5caebf890c None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:16.127039 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-1a13472d-c25a-4088-af11-4a5caebf890c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:16:16.127039 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:16.127039 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:16.127039 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:16.127039 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:16.127039 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:16.127039 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:16.127039 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:16.127039 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:16:16.127039 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:16.127039 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:16.127039 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:16.127039 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:16.127039 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:16.127039 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:16.127039 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:16.127039 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:16.128202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:16.128202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:16.128202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:16.128202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:16.128202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:16.128202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:16.128202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:16.128202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:16.128202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:16.128202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:16.128202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:16.128202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:16.128202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:16.128202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:16.128202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:16.128202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:16.129119 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:16.129119 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:16.129119 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:16.129119 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:16.129119 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:16.129119 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:16.129119 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:16.129119 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:16.129119 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:16.129119 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:16.129119 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:16.129119 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:16.129119 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:16:16.129119 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:16.129119 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 335/679] 10.222.0.30 () {56 vars in 1126 bytes} [Wed Nov 2 07:16:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:16.129911 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808961d5-dddc-4174-abec-f338a35d82a2 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:16.129911 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808961d5-dddc-4174-abec-f338a35d82a2 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:16.130110 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808961d5-dddc-4174-abec-f338a35d82a2 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:16.131493 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 336/680] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:16.134298 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808961d5-dddc-4174-abec-f338a35d82a2 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:16.152805 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0cba3b-cdff-4d67-bebd-4e979611051d None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:16.186766 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd93c85-98e5-4559-b7a1-000605badbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:16.187381 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd93c85-98e5-4559-b7a1-000605badbf1 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:16.187653 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd93c85-98e5-4559-b7a1-000605badbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:16.187923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd93c85-98e5-4559-b7a1-000605badbf1 None admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:16.188570 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd93c85-98e5-4559-b7a1-000605badbf1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:16.188866 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd93c85-98e5-4559-b7a1-000605badbf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:16.190226 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd93c85-98e5-4559-b7a1-000605badbf1 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:16.198438 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 345/681] 10.222.0.30 () {56 vars in 1115 bytes} [Wed Nov 2 07:16:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:16.204137 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9cb41-4386-4e3b-89aa-df7c9048f177 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:16.204415 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9cb41-4386-4e3b-89aa-df7c9048f177 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:16.204686 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9cb41-4386-4e3b-89aa-df7c9048f177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:16.222548 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-71b9cb41-4386-4e3b-89aa-df7c9048f177 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:16.223783 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-71b9cb41-4386-4e3b-89aa-df7c9048f177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:16.241139 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:16.254212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808961d5-dddc-4174-abec-f338a35d82a2 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:16.292174 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 346/682] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:16.293124 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9cb41-4386-4e3b-89aa-df7c9048f177 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:16.303527 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e3746e-08a2-4e33-b5fb-d4dc92505de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:16.304424 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3746e-08a2-4e33-b5fb-d4dc92505de1 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:16.304808 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3746e-08a2-4e33-b5fb-d4dc92505de1 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:16.305224 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3746e-08a2-4e33-b5fb-d4dc92505de1 None admin] PATH_INFO: `/v3/services` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:16.305970 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e3746e-08a2-4e33-b5fb-d4dc92505de1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:16.306355 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e3746e-08a2-4e33-b5fb-d4dc92505de1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:16.307881 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e3746e-08a2-4e33-b5fb-d4dc92505de1 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:16.318625 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 337/683] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Nov 2 07:16:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:16.320122 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e3746e-08a2-4e33-b5fb-d4dc92505de1 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:16.334888 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:16.421255 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9cb41-4386-4e3b-89aa-df7c9048f177 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:16.440686 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e3746e-08a2-4e33-b5fb-d4dc92505de1 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:16.464382 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c3bb7-0ad2-42ce-82d6-c304ec4daf84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:16.466256 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c3bb7-0ad2-42ce-82d6-c304ec4daf84 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:16.466993 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c3bb7-0ad2-42ce-82d6-c304ec4daf84 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:16.467492 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c3bb7-0ad2-42ce-82d6-c304ec4daf84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:16.478631 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c3bb7-0ad2-42ce-82d6-c304ec4daf84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:16.480697 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c3bb7-0ad2-42ce-82d6-c304ec4daf84 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:16.482528 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e133b-d548-4039-a6ae-bf6eca09a7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:16.483276 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e133b-d548-4039-a6ae-bf6eca09a7d3 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:16.483547 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e133b-d548-4039-a6ae-bf6eca09a7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:16.483805 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e133b-d548-4039-a6ae-bf6eca09a7d3 None admin] PATH_INFO: `/v3/services` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:16.484520 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e133b-d548-4039-a6ae-bf6eca09a7d3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:16.484819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e133b-d548-4039-a6ae-bf6eca09a7d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:16.486161 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e133b-d548-4039-a6ae-bf6eca09a7d3 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:16.496103 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 338/684] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Nov 2 07:16:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:16.499616 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e133b-d548-4039-a6ae-bf6eca09a7d3 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:16.500860 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-b31c3bb7-0ad2-42ce-82d6-c304ec4daf84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:16.500860 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:16.500860 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:16.500860 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:16.500860 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:16.500860 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:16.500860 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:16.500860 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:16.500860 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:16.500860 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:16.500860 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:16.500860 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:16.500860 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:16:16.500860 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:16:16.500860 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:16:16.500860 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:16:16.501803 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:16.501803 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:16.501803 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:16.501803 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:16.501803 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:16.501803 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:16.501803 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:16.501803 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:16.501803 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:16.501803 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:16.501803 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:16.501803 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:16.501803 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:16.501803 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:16.501803 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:16.501803 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:16.501803 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:16:16.502704 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:16:16.502704 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:16.502704 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:16.506862 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 347/685] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:16.507689 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c3bb7-0ad2-42ce-82d6-c304ec4daf84 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:16.528709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:16.608889 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:16.619347 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e133b-d548-4039-a6ae-bf6eca09a7d3 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:16.632697 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c3bb7-0ad2-42ce-82d6-c304ec4daf84 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:16.654213 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b7509-d854-4794-ae00-41b19f80d070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:16.655138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b7509-d854-4794-ae00-41b19f80d070 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:16.655423 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b7509-d854-4794-ae00-41b19f80d070 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:16.655686 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b7509-d854-4794-ae00-41b19f80d070 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:16.656378 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b7509-d854-4794-ae00-41b19f80d070 None admin] RBAC: Adding query filter params (interface=public, service_id=4bf1686c92ef4b4d9b5ea9e67086a1f7, region_id=RegionOne) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:16.656679 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b7509-d854-4794-ae00-41b19f80d070 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:16.658102 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b7509-d854-4794-ae00-41b19f80d070 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:16.663205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 339/686] 10.222.0.30 () {56 vars in 1251 bytes} [Wed Nov 2 07:16:16 2022] GET /identity/v3/endpoints?service_id=4bf1686c92ef4b4d9b5ea9e67086a1f7&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:16.668907 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b7509-d854-4794-ae00-41b19f80d070 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:16.669338 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5bbbe7-bf05-472a-952e-ce536e031b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:16.669947 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5bbbe7-bf05-472a-952e-ce536e031b65 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:16.670218 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5bbbe7-bf05-472a-952e-ce536e031b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:16.670482 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5bbbe7-bf05-472a-952e-ce536e031b65 None admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:16.671141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5bbbe7-bf05-472a-952e-ce536e031b65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:16.671500 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5bbbe7-bf05-472a-952e-ce536e031b65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:16.672903 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5bbbe7-bf05-472a-952e-ce536e031b65 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:16.689958 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 348/687] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:16.689958 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5bbbe7-bf05-472a-952e-ce536e031b65 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:16.768730 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:16.789872 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b7509-d854-4794-ae00-41b19f80d070 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:16.791604 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5bbbe7-bf05-472a-952e-ce536e031b65 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:16.824756 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f85de-4166-49d3-9be1-552a9b52b8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:16.825363 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f85de-4166-49d3-9be1-552a9b52b8e8 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:16.825639 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f85de-4166-49d3-9be1-552a9b52b8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:16.825907 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f85de-4166-49d3-9be1-552a9b52b8e8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:16.826489 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6f85de-4166-49d3-9be1-552a9b52b8e8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:16.827461 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de7c00-4efa-48e7-aeeb-e5eac8bbe30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:16.827892 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6f85de-4166-49d3-9be1-552a9b52b8e8 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:16.828077 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de7c00-4efa-48e7-aeeb-e5eac8bbe30e None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:16.828355 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de7c00-4efa-48e7-aeeb-e5eac8bbe30e None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:16.828629 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de7c00-4efa-48e7-aeeb-e5eac8bbe30e None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:16.829230 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de7c00-4efa-48e7-aeeb-e5eac8bbe30e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:16.830604 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de7c00-4efa-48e7-aeeb-e5eac8bbe30e None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:16.837510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-4f6f85de-4166-49d3-9be1-552a9b52b8e8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:16:16.837510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:16.837510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:16.837510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:16.837510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:16.837510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:16.837510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:16.837510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:16.837510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:16:16.837510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:16.837510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:16.837510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:16.837510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:16.837510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:16.837510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:16.837510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:16.838553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:16.838553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:16.838553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:16.838553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:16.838553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:16.838553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:16.838553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:16.838553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:16.838553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:16.838553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:16.838553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:16.838553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:16.838553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:16.838553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:16.838553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:16.838553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:16.838553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:16.839649 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:16.839649 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:16.839649 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:16.839649 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:16.839649 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:16.839649 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:16.839649 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:16.839649 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:16.839649 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:16.839649 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:16.839649 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:16.839649 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:16.839649 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 07:16:16.839649 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:16.841805 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 340/688] 10.222.0.30 () {56 vars in 1132 bytes} [Wed Nov 2 07:16:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:16.842672 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f85de-4166-49d3-9be1-552a9b52b8e8 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:16.848002 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 349/689] 10.222.0.30 () {56 vars in 1087 bytes} [Wed Nov 2 07:16:16 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:16:16.857416 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de7c00-4efa-48e7-aeeb-e5eac8bbe30e None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:16.867348 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:16.971383 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f85de-4166-49d3-9be1-552a9b52b8e8 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:16.978376 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de7c00-4efa-48e7-aeeb-e5eac8bbe30e None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:17.015807 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f1893-e350-4865-85ad-20cd53260e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:17.016733 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f1893-e350-4865-85ad-20cd53260e65 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:17.017016 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f1893-e350-4865-85ad-20cd53260e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:17.017282 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f1893-e350-4865-85ad-20cd53260e65 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:17.022145 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67020848-4dae-4f2c-af43-876193918155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:17.023490 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67020848-4dae-4f2c-af43-876193918155 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:17.024102 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67020848-4dae-4f2c-af43-876193918155 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:17.024762 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67020848-4dae-4f2c-af43-876193918155 None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:17.025761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67020848-4dae-4f2c-af43-876193918155 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:17.026615 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67020848-4dae-4f2c-af43-876193918155 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:17.026787 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f1893-e350-4865-85ad-20cd53260e65 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:17.028054 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67020848-4dae-4f2c-af43-876193918155 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:17.028163 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f1893-e350-4865-85ad-20cd53260e65 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:17.038457 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 350/690] 10.222.0.30 () {56 vars in 1126 bytes} [Wed Nov 2 07:16:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:17.039367 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67020848-4dae-4f2c-af43-876193918155 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:17.040544 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-9a2f1893-e350-4865-85ad-20cd53260e65 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 07:16:17.040544 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:17.040544 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:17.040544 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:17.040544 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:17.040544 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:17.040544 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:17.040544 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:17.040544 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:17.040544 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:17.040544 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:17.040544 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:17.040544 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:16:17.040544 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:16:17.040544 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:16:17.040544 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:16:17.041480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:17.041480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:17.041480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:16:17.041480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:16:17.041480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:16:17.041480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:16:17.041480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:17.041480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:17.041480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:17.041480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:17.041480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:17.041480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:17.041480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:17.041480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:17.041480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:17.041480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:17.041480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:17.041480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:17.042446 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:17.042446 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:17.042446 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:16:17.042446 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:16:17.042446 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:16:17.042446 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:16:17.042446 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:16:17.042446 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:16:17.042446 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 07:16:17.042446 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:17.042446 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 341/691] 10.222.0.30 () {56 vars in 1117 bytes} [Wed Nov 2 07:16:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:17.047587 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f1893-e350-4865-85ad-20cd53260e65 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:17.060303 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:17.126843 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:17.162305 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67020848-4dae-4f2c-af43-876193918155 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:17.173654 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f1893-e350-4865-85ad-20cd53260e65 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:17.200642 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164cae97-7c93-4a72-82e8-4ed9da1df4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:17.201255 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cae97-7c93-4a72-82e8-4ed9da1df4bb None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:17.201533 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cae97-7c93-4a72-82e8-4ed9da1df4bb None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:17.201804 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cae97-7c93-4a72-82e8-4ed9da1df4bb None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:17.202431 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164cae97-7c93-4a72-82e8-4ed9da1df4bb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:17.202759 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164cae97-7c93-4a72-82e8-4ed9da1df4bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:17.204153 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164cae97-7c93-4a72-82e8-4ed9da1df4bb None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:17.208011 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf27c8e-0133-4466-8da1-54bb56311314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:17.208803 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf27c8e-0133-4466-8da1-54bb56311314 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:17.209120 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 351/692] 10.222.0.30 () {56 vars in 1126 bytes} [Wed Nov 2 07:16:17 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:17.209383 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf27c8e-0133-4466-8da1-54bb56311314 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:17.209776 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf27c8e-0133-4466-8da1-54bb56311314 None admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:17.210378 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712158ef-b9ff-4b37-8eb6-6efa56c6bb1d None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:17.210674 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712158ef-b9ff-4b37-8eb6-6efa56c6bb1d None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:17.210929 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf27c8e-0133-4466-8da1-54bb56311314 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:17.211027 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712158ef-b9ff-4b37-8eb6-6efa56c6bb1d None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:17.211461 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf27c8e-0133-4466-8da1-54bb56311314 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:17.213029 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf27c8e-0133-4466-8da1-54bb56311314 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:17.216722 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 352/693] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:17.217465 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712158ef-b9ff-4b37-8eb6-6efa56c6bb1d None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:17.233007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 342/694] 10.222.0.30 () {56 vars in 1149 bytes} [Wed Nov 2 07:16:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:17.234620 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb78892-f252-4067-9b64-8ad1ff4961ba None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:17.234934 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb78892-f252-4067-9b64-8ad1ff4961ba None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:17.235202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb78892-f252-4067-9b64-8ad1ff4961ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:17.253833 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-feb78892-f252-4067-9b64-8ad1ff4961ba None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:17.255082 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-feb78892-f252-4067-9b64-8ad1ff4961ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:17.319654 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:17.334959 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 343/695] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:17.334959 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb78892-f252-4067-9b64-8ad1ff4961ba None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:17.336870 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712158ef-b9ff-4b37-8eb6-6efa56c6bb1d None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:17.379367 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe252aa-eb81-448c-b664-7646cc5dd30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:17.380497 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe252aa-eb81-448c-b664-7646cc5dd30a None admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:17.381463 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe252aa-eb81-448c-b664-7646cc5dd30a None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:17.381920 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe252aa-eb81-448c-b664-7646cc5dd30a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:17.383058 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe252aa-eb81-448c-b664-7646cc5dd30a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:17.385274 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe252aa-eb81-448c-b664-7646cc5dd30a None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:17.393904 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:17.415238 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 353/696] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:16:17 2022] POST /identity/v3/endpoints => generated 347 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:17.455892 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb78892-f252-4067-9b64-8ad1ff4961ba None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:17.507617 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f8b1f-8178-4360-86c7-873cb87eec19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:17.508802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f8b1f-8178-4360-86c7-873cb87eec19 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:17.509246 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f8b1f-8178-4360-86c7-873cb87eec19 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:17.509660 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f8b1f-8178-4360-86c7-873cb87eec19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:17.578267 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:17.590289 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f8b1f-8178-4360-86c7-873cb87eec19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:17.592623 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f8b1f-8178-4360-86c7-873cb87eec19 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:17.619445 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe252aa-eb81-448c-b664-7646cc5dd30a None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:17.715927 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 344/697] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 07:16:17 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:16:17.723681 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f8b1f-8178-4360-86c7-873cb87eec19 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:17.758144 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe252aa-eb81-448c-b664-7646cc5dd30a None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:17.807114 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b181a01-b59e-4be0-8033-24232c841d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:17.807869 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b181a01-b59e-4be0-8033-24232c841d20 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:17.808229 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b181a01-b59e-4be0-8033-24232c841d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:17.808563 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b181a01-b59e-4be0-8033-24232c841d20 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:17.809216 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b181a01-b59e-4be0-8033-24232c841d20 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:17.811530 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b181a01-b59e-4be0-8033-24232c841d20 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:17.824446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-9b181a01-b59e-4be0-8033-24232c841d20 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:16:17.824446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:17.824446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:17.824446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:17.824446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:17.824446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:17.824446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:17.824446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:17.824446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:16:17.824446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:17.824446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:17.824446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:17.824446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:17.824446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:17.824446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:17.824446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:17.824446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:17.825603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:17.825603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:17.825603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:17.825603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:17.825603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:17.825603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:17.825603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:17.825603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:17.825603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:17.825603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:17.825603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:17.825603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:17.825603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:17.825603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:17.825603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:17.825603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:17.826425 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:17.826425 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:17.826425 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:17.826425 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:17.826425 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:17.826425 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:17.826425 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:17.826425 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:17.826425 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:17.826425 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:17.826425 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:17.826425 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:17.826425 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:16:17.826425 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:17.827222 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 354/698] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Nov 2 07:16:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:17.833015 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b181a01-b59e-4be0-8033-24232c841d20 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:17.844270 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:17.856097 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f8b1f-8178-4360-86c7-873cb87eec19 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:17.904189 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f764af-5704-448a-8fd9-d28185b3982a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:17.904960 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f764af-5704-448a-8fd9-d28185b3982a None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:17.905293 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f764af-5704-448a-8fd9-d28185b3982a None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:17.905626 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f764af-5704-448a-8fd9-d28185b3982a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:17.910003 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:17.922595 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f764af-5704-448a-8fd9-d28185b3982a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:17.924181 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f764af-5704-448a-8fd9-d28185b3982a None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:17.942541 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-73f764af-5704-448a-8fd9-d28185b3982a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:17.942541 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:17.942541 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:17.942541 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:17.942541 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:17.942541 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:17.942541 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:17.942541 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:17.942541 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:17.942541 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:17.942541 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:17.942541 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:17.942541 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:16:17.942541 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:16:17.942541 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:16:17.942541 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:16:17.943571 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:17.943571 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:17.943571 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:17.943571 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:17.943571 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:17.943571 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:17.943571 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:17.943571 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:17.943571 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:17.943571 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:17.943571 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:17.943571 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:17.943571 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:17.943571 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:17.943571 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:17.943571 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:17.943571 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:16:17.944471 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:16:17.944471 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:17.944471 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:17.944471 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 345/699] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:17.951092 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f764af-5704-448a-8fd9-d28185b3982a None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:17.974851 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b181a01-b59e-4be0-8033-24232c841d20 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:18.017566 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941645d5-83af-4462-9757-a7b78629e6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:18.018985 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941645d5-83af-4462-9757-a7b78629e6bb None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:18.019415 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941645d5-83af-4462-9757-a7b78629e6bb None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:18.019851 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941645d5-83af-4462-9757-a7b78629e6bb None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:18.020619 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941645d5-83af-4462-9757-a7b78629e6bb None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:18.021062 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941645d5-83af-4462-9757-a7b78629e6bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:18.023256 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941645d5-83af-4462-9757-a7b78629e6bb None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:18.029774 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 355/700] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Nov 2 07:16:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:18.037053 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941645d5-83af-4462-9757-a7b78629e6bb None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:18.067515 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f764af-5704-448a-8fd9-d28185b3982a None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:18.104651 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde6c9f-e597-4128-8d73-60dbbe33c455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:18.105256 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde6c9f-e597-4128-8d73-60dbbe33c455 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:18.105530 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde6c9f-e597-4128-8d73-60dbbe33c455 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:18.105794 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde6c9f-e597-4128-8d73-60dbbe33c455 None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:18.106524 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde6c9f-e597-4128-8d73-60dbbe33c455 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:18.106865 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde6c9f-e597-4128-8d73-60dbbe33c455 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:18.108239 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde6c9f-e597-4128-8d73-60dbbe33c455 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:18.121220 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 346/701] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:16:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:18.130693 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde6c9f-e597-4128-8d73-60dbbe33c455 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:18.142581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:18.166364 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941645d5-83af-4462-9757-a7b78629e6bb None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:18.191121 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:18.207246 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71f3a0-7eaf-4471-ae5a-1783ee444fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:18.207866 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71f3a0-7eaf-4471-ae5a-1783ee444fb1 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:18.208146 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71f3a0-7eaf-4471-ae5a-1783ee444fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:18.208399 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71f3a0-7eaf-4471-ae5a-1783ee444fb1 None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:18.209029 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d71f3a0-7eaf-4471-ae5a-1783ee444fb1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:18.209325 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d71f3a0-7eaf-4471-ae5a-1783ee444fb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:18.210750 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d71f3a0-7eaf-4471-ae5a-1783ee444fb1 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:18.216426 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 356/702] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Nov 2 07:16:18 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:18.222353 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71f3a0-7eaf-4471-ae5a-1783ee444fb1 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:18.234785 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde6c9f-e597-4128-8d73-60dbbe33c455 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:18.270511 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d46b47-66a7-49d9-8182-d02bf9e2fbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:18.271099 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d46b47-66a7-49d9-8182-d02bf9e2fbf1 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:18.271376 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d46b47-66a7-49d9-8182-d02bf9e2fbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:18.271645 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d46b47-66a7-49d9-8182-d02bf9e2fbf1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:18.279286 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d46b47-66a7-49d9-8182-d02bf9e2fbf1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:18.280806 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d46b47-66a7-49d9-8182-d02bf9e2fbf1 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:18.292060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-49d46b47-66a7-49d9-8182-d02bf9e2fbf1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:16:18.292060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:18.292060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:18.292060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:18.292060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:18.292060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:18.292060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:18.292060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:18.292060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:18.292060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:18.292060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:18.292060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:18.292060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:16:18.292060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:16:18.292060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:16:18.293141 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:16:18.293141 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:18.293141 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:18.293141 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:18.293141 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:18.293141 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:18.293141 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:18.293141 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:18.293141 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:18.293141 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:18.293141 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:18.293141 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:18.293141 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:18.293141 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:18.293141 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:18.293141 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:18.293141 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:18.294258 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:16:18.294258 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:16:18.294258 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:16:18.294258 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:16:18.294258 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:16:18.294258 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:16:18.294258 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:16:18.294258 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:18.298278 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 347/703] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:16:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:18.299539 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d46b47-66a7-49d9-8182-d02bf9e2fbf1 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:18.337530 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71f3a0-7eaf-4471-ae5a-1783ee444fb1 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:18.364314 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:18.383667 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4effe530-3558-4bc0-b3de-bd3d6b33e793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:18.384582 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4effe530-3558-4bc0-b3de-bd3d6b33e793 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:18.384863 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4effe530-3558-4bc0-b3de-bd3d6b33e793 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:18.385131 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4effe530-3558-4bc0-b3de-bd3d6b33e793 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:18.386089 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4effe530-3558-4bc0-b3de-bd3d6b33e793 None admin] RBAC: Adding query filter params (interface=public, service_id=4cd85d6b12324dada1d233d63000af3a, region_id=RegionOne) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:18.386416 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4effe530-3558-4bc0-b3de-bd3d6b33e793 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:18.388138 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4effe530-3558-4bc0-b3de-bd3d6b33e793 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:18.400200 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 357/704] 10.222.0.30 () {56 vars in 1251 bytes} [Wed Nov 2 07:16:18 2022] GET /identity/v3/endpoints?service_id=4cd85d6b12324dada1d233d63000af3a&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:18.408687 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4effe530-3558-4bc0-b3de-bd3d6b33e793 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:18.412909 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d46b47-66a7-49d9-8182-d02bf9e2fbf1 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:18.449363 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b3101e-88b8-412d-a578-a7e44650c78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:18.449978 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b3101e-88b8-412d-a578-a7e44650c78c None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:18.450253 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b3101e-88b8-412d-a578-a7e44650c78c None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:18.450517 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b3101e-88b8-412d-a578-a7e44650c78c None admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:18.451249 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b3101e-88b8-412d-a578-a7e44650c78c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:18.451551 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b3101e-88b8-412d-a578-a7e44650c78c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:18.452987 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b3101e-88b8-412d-a578-a7e44650c78c None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:18.466027 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:18.470912 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 348/705] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 07:16:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:18.472331 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b3101e-88b8-412d-a578-a7e44650c78c None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:18.540840 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4effe530-3558-4bc0-b3de-bd3d6b33e793 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:18.582610 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b3101e-88b8-412d-a578-a7e44650c78c None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:18.588571 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7830718-9ff4-491e-9452-6a8ab8c504ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:18.589390 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7830718-9ff4-491e-9452-6a8ab8c504ef None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:18.589813 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7830718-9ff4-491e-9452-6a8ab8c504ef None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:18.590182 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7830718-9ff4-491e-9452-6a8ab8c504ef None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:18.590986 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7830718-9ff4-491e-9452-6a8ab8c504ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:18.599816 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7830718-9ff4-491e-9452-6a8ab8c504ef None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:18.614230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:18.618718 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 358/706] 10.222.0.30 () {56 vars in 1087 bytes} [Wed Nov 2 07:16:18 2022] GET /identity/v3/services => generated 1497 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:16:18.632982 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3b2bc-6018-4d90-aa6c-a51fe67f7876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:18.634528 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3b2bc-6018-4d90-aa6c-a51fe67f7876 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:18.635007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3b2bc-6018-4d90-aa6c-a51fe67f7876 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:18.635449 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3b2bc-6018-4d90-aa6c-a51fe67f7876 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:18.636369 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e3b2bc-6018-4d90-aa6c-a51fe67f7876 None admin] RBAC: Adding query filter params (role.id=e6483a4fca514243bed809857f555411, scope.project.id=25e2bc8417754f909b416b93d5b7afb7, user.id=364555ec385c4dbb91907fb9c5116d2a) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:18.636822 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e3b2bc-6018-4d90-aa6c-a51fe67f7876 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:18.638433 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e3b2bc-6018-4d90-aa6c-a51fe67f7876 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:18.651182 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 349/707] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 07:16:18 2022] GET /identity/v3/role_assignments?user.id=364555ec385c4dbb91907fb9c5116d2a&scope.project.id=25e2bc8417754f909b416b93d5b7afb7&role.id=e6483a4fca514243bed809857f555411 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:19.050704 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682d57e-4420-4b81-b586-29f56f846bc7 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:19.051257 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682d57e-4420-4b81-b586-29f56f846bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:19.051257 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682d57e-4420-4b81-b586-29f56f846bc7 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:19.052876 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 359/708] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:19.058435 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024f084-40c2-465d-a375-d9f0f6caac3e None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:19.058967 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024f084-40c2-465d-a375-d9f0f6caac3e None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:19.059430 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024f084-40c2-465d-a375-d9f0f6caac3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:19.078561 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-4024f084-40c2-465d-a375-d9f0f6caac3e None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:19.080548 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-4024f084-40c2-465d-a375-d9f0f6caac3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:19.156320 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 350/709] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:16:19.420660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682d57e-4420-4b81-b586-29f56f846bc7 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:19.528738 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682d57e-4420-4b81-b586-29f56f846bc7 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:19.540002 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:19.565886 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3b3dd-7e1b-4044-8971-99afa8dd2909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:19.566540 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3b3dd-7e1b-4044-8971-99afa8dd2909 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:19.566848 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3b3dd-7e1b-4044-8971-99afa8dd2909 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:19.567117 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3b3dd-7e1b-4044-8971-99afa8dd2909 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:19.567705 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b3b3dd-7e1b-4044-8971-99afa8dd2909 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:19.569089 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b3b3dd-7e1b-4044-8971-99afa8dd2909 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:19.578586 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-e2b3b3dd-7e1b-4044-8971-99afa8dd2909 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:16:19.578586 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:19.578586 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:19.578586 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:19.578586 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:19.578586 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:19.578586 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:19.578586 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:19.578586 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:16:19.578586 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:19.578586 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:19.578586 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:19.578586 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:19.578586 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:19.578586 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:19.578586 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:19.578586 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:19.579624 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:19.579624 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:19.579624 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:19.579624 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:19.579624 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:19.579624 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:19.579624 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:19.579624 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:19.579624 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:19.579624 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:19.579624 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:19.579624 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:19.579624 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:19.579624 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:19.579624 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:19.579624 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:19.580470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:19.580470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:19.580470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:19.580470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:19.580470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:19.580470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:19.580470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:19.580470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:19.580470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:19.580470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:19.580470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:19.580470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:19.580470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:16:19.580470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:19.580470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 360/710] 10.222.0.30 () {56 vars in 1126 bytes} [Wed Nov 2 07:16:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:19.586407 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024f084-40c2-465d-a375-d9f0f6caac3e None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:19.692352 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024f084-40c2-465d-a375-d9f0f6caac3e None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:19.709803 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:19.726720 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb52ce7-2369-4178-a165-e0470ff9c9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:19.727369 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb52ce7-2369-4178-a165-e0470ff9c9ff None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:19.727697 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb52ce7-2369-4178-a165-e0470ff9c9ff None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:19.727966 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb52ce7-2369-4178-a165-e0470ff9c9ff None admin] PATH_INFO: `/v3/services` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:19.728581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb52ce7-2369-4178-a165-e0470ff9c9ff None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:19.728876 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb52ce7-2369-4178-a165-e0470ff9c9ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:19.730254 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb52ce7-2369-4178-a165-e0470ff9c9ff None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:19.735362 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 351/711] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Nov 2 07:16:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:19.741101 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3b3dd-7e1b-4044-8971-99afa8dd2909 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:19.743277 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f9481-fbbb-4170-8f6d-2a3498919260 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:19.743549 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f9481-fbbb-4170-8f6d-2a3498919260 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:19.743814 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f9481-fbbb-4170-8f6d-2a3498919260 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:19.745292 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 352/712] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:19.752331 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd97051-189c-444f-a7c8-6fcfa5c0d68f None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:19.752769 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd97051-189c-444f-a7c8-6fcfa5c0d68f None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:19.753197 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd97051-189c-444f-a7c8-6fcfa5c0d68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:19.777088 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-1cd97051-189c-444f-a7c8-6fcfa5c0d68f None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:19.778767 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-1cd97051-189c-444f-a7c8-6fcfa5c0d68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:19.866160 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3b3dd-7e1b-4044-8971-99afa8dd2909 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:19.866363 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 353/713] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:19.903176 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe92f5ee-b7dc-4526-9591-36a551226457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:19.904130 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92f5ee-b7dc-4526-9591-36a551226457 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:19.904592 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92f5ee-b7dc-4526-9591-36a551226457 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:19.905047 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92f5ee-b7dc-4526-9591-36a551226457 None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:19.905858 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe92f5ee-b7dc-4526-9591-36a551226457 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:19.906364 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe92f5ee-b7dc-4526-9591-36a551226457 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:19.907962 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe92f5ee-b7dc-4526-9591-36a551226457 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:19.913626 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 361/714] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Nov 2 07:16:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:19.919576 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd97051-189c-444f-a7c8-6fcfa5c0d68f None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:20.020282 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd97051-189c-444f-a7c8-6fcfa5c0d68f None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:20.043927 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:20.056286 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ca606-b1c4-433e-9b36-d4ec0039c896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:20.057164 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ca606-b1c4-433e-9b36-d4ec0039c896 None admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:20.057647 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ca606-b1c4-433e-9b36-d4ec0039c896 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:20.058100 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ca606-b1c4-433e-9b36-d4ec0039c896 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:20.059085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1ca606-b1c4-433e-9b36-d4ec0039c896 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:20.060632 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1ca606-b1c4-433e-9b36-d4ec0039c896 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:20.085232 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 354/715] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:16:19 2022] POST /identity/v3/endpoints => generated 349 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:16:20.139269 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe92f5ee-b7dc-4526-9591-36a551226457 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:20.188142 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:20.258740 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe92f5ee-b7dc-4526-9591-36a551226457 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:20.304095 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6dd889-14c2-41e3-bae3-b41e89618dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:20.304724 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6dd889-14c2-41e3-bae3-b41e89618dba None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:20.305002 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6dd889-14c2-41e3-bae3-b41e89618dba None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:20.305353 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6dd889-14c2-41e3-bae3-b41e89618dba None admin] PATH_INFO: `/v3/services/compute` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:20.305941 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6dd889-14c2-41e3-bae3-b41e89618dba None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:20.307426 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6dd889-14c2-41e3-bae3-b41e89618dba None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:20.316435 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-1d6dd889-14c2-41e3-bae3-b41e89618dba None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:16:20.316435 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:20.316435 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:20.316435 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:20.316435 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:20.316435 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:20.316435 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:20.316435 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:20.316435 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:16:20.316435 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:20.316435 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:20.316435 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:20.316435 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:20.316435 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:20.316435 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:20.316435 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:20.316435 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:20.317414 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:20.317414 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:20.317414 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:20.317414 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:20.317414 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:20.317414 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:20.317414 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:20.317414 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:20.317414 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:20.317414 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:20.317414 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:20.317414 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:20.317414 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:20.317414 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:20.317414 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:20.317414 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:20.318263 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:20.318263 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:20.318263 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:20.318263 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:20.318263 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:20.318263 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:20.318263 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:20.318263 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:20.318263 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:20.318263 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:20.318263 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:20.318263 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:20.318263 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:16:20.318263 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:20.318263 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 362/716] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:16:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:20.324256 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ca606-b1c4-433e-9b36-d4ec0039c896 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:20.428206 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ca606-b1c4-433e-9b36-d4ec0039c896 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:20.463208 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36a9eb-40a1-44bf-aa8f-8052061918d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:20.464166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36a9eb-40a1-44bf-aa8f-8052061918d8 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:20.464615 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36a9eb-40a1-44bf-aa8f-8052061918d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:20.465078 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36a9eb-40a1-44bf-aa8f-8052061918d8 None admin] PATH_INFO: `/v3/services` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:20.465890 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff36a9eb-40a1-44bf-aa8f-8052061918d8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:20.466382 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff36a9eb-40a1-44bf-aa8f-8052061918d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:20.468003 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff36a9eb-40a1-44bf-aa8f-8052061918d8 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:20.473555 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 355/717] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Nov 2 07:16:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:20.479558 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6dd889-14c2-41e3-bae3-b41e89618dba None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:20.582925 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:20.587094 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6dd889-14c2-41e3-bae3-b41e89618dba None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:20.622706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c9967-d4da-4c6f-8a8f-9b7f0d10f8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:20.623324 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c9967-d4da-4c6f-8a8f-9b7f0d10f8bb None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:20.623602 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c9967-d4da-4c6f-8a8f-9b7f0d10f8bb None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:20.623870 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c9967-d4da-4c6f-8a8f-9b7f0d10f8bb None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:20.624498 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c9967-d4da-4c6f-8a8f-9b7f0d10f8bb None admin] RBAC: Adding query filter params (type=compute) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:20.624796 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c9967-d4da-4c6f-8a8f-9b7f0d10f8bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:20.626175 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c9967-d4da-4c6f-8a8f-9b7f0d10f8bb None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:20.631148 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 363/718] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Nov 2 07:16:20 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:20.894647 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0390f42-a518-41dd-809b-f056fe51f6f7 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:20.895025 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0390f42-a518-41dd-809b-f056fe51f6f7 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:20.895221 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0390f42-a518-41dd-809b-f056fe51f6f7 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:20.896785 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 356/719] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:20.902790 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b1346-8cfe-4de3-b2e5-c0517ccbbb7f None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:20.903074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b1346-8cfe-4de3-b2e5-c0517ccbbb7f None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:20.903346 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b1346-8cfe-4de3-b2e5-c0517ccbbb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:20.921735 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-c19b1346-8cfe-4de3-b2e5-c0517ccbbb7f None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:20.923022 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-c19b1346-8cfe-4de3-b2e5-c0517ccbbb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:20.969810 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d5d98-0fcd-4344-828a-67d8af23145a None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:20.970081 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d5d98-0fcd-4344-828a-67d8af23145a None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:20.970350 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d5d98-0fcd-4344-828a-67d8af23145a None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:20.971851 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 357/720] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:20.979812 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7e7e71-692b-4a0a-876b-331aa0488690 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:20.980391 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7e7e71-692b-4a0a-876b-331aa0488690 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:20.980852 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7e7e71-692b-4a0a-876b-331aa0488690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:20.996168 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 364/721] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:21.003846 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-cc7e7e71-692b-4a0a-876b-331aa0488690 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:21.005272 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-cc7e7e71-692b-4a0a-876b-331aa0488690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:21.033909 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:21.082808 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 358/722] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:16:21.262441 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b1346-8cfe-4de3-b2e5-c0517ccbbb7f None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:21.353116 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:21.361697 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b1346-8cfe-4de3-b2e5-c0517ccbbb7f None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:21.370049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7e7e71-692b-4a0a-876b-331aa0488690 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:21.407431 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad92f4-9944-4135-868c-f1ade892d37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:21.408981 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad92f4-9944-4135-868c-f1ade892d37b None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:21.409430 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad92f4-9944-4135-868c-f1ade892d37b None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:21.409897 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad92f4-9944-4135-868c-f1ade892d37b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:21.410808 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad92f4-9944-4135-868c-f1ade892d37b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:21.412218 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad92f4-9944-4135-868c-f1ade892d37b None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:21.423177 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-88ad92f4-9944-4135-868c-f1ade892d37b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:16:21.423177 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:21.423177 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:21.423177 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:21.423177 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:21.423177 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:21.423177 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:21.423177 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:21.423177 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:16:21.423177 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:21.423177 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:21.423177 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:21.423177 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:21.423177 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:21.423177 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:21.423177 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:21.423177 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:21.424210 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:21.424210 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:21.424210 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:21.424210 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:21.424210 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:21.424210 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:21.424210 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:21.424210 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:21.424210 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:21.424210 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:21.424210 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:21.424210 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:21.424210 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:21.424210 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:21.424210 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:21.424210 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:21.425017 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:21.425017 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:21.425017 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:21.425017 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:21.425017 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:21.425017 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:21.425017 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:21.425017 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:21.425017 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:21.425017 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:21.425017 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:21.425017 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:21.425017 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 07:16:21.425017 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:21.425017 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 365/723] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Nov 2 07:16:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:21.431263 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad92f4-9944-4135-868c-f1ade892d37b None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:21.490534 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7e7e71-692b-4a0a-876b-331aa0488690 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:21.536359 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f03494-5139-4fd0-a3e9-351e313a2546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:21.537439 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f03494-5139-4fd0-a3e9-351e313a2546 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:21.537905 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f03494-5139-4fd0-a3e9-351e313a2546 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:21.538415 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f03494-5139-4fd0-a3e9-351e313a2546 None admin] PATH_INFO: `/v3/services/image` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:21.540669 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f03494-5139-4fd0-a3e9-351e313a2546 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:21.542590 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f03494-5139-4fd0-a3e9-351e313a2546 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:21.553357 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad92f4-9944-4135-868c-f1ade892d37b None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:21.554007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-28f03494-5139-4fd0-a3e9-351e313a2546 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:16:21.554007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:21.554007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:21.554007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:21.554007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:21.554007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:21.554007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:21.554007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:21.554007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:16:21.554007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:21.554007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:21.554007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:21.554007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:21.554007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:21.554007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:21.554007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:21.554007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:21.556186 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:21.556186 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:21.556186 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:21.556186 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:21.556186 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:21.556186 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:21.556186 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:21.556186 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:21.556186 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:21.556186 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:21.556186 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:21.556186 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:21.556186 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:21.556186 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:21.556186 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:21.556186 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:21.557244 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:21.557244 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:21.557244 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:21.557244 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:21.557244 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:21.557244 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:21.557244 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:21.557244 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:21.557244 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:21.557244 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:21.557244 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:21.557244 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:21.557244 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:16:21.557244 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:21.566042 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 359/724] 10.222.0.30 () {56 vars in 1105 bytes} [Wed Nov 2 07:16:21 2022] GET /identity/v3/services/image => generated 86 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:21.567443 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f03494-5139-4fd0-a3e9-351e313a2546 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:21.594459 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0097fa-daec-4b0e-aa71-db4e3f3ded13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:21.595266 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0097fa-daec-4b0e-aa71-db4e3f3ded13 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:21.595984 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0097fa-daec-4b0e-aa71-db4e3f3ded13 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:21.595984 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0097fa-daec-4b0e-aa71-db4e3f3ded13 None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:21.596704 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0097fa-daec-4b0e-aa71-db4e3f3ded13 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:21.597093 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0097fa-daec-4b0e-aa71-db4e3f3ded13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:21.598669 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0097fa-daec-4b0e-aa71-db4e3f3ded13 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:21.603812 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:21.604646 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 366/725] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Nov 2 07:16:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:21.615584 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0097fa-daec-4b0e-aa71-db4e3f3ded13 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:21.640289 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:21.699513 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f03494-5139-4fd0-a3e9-351e313a2546 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:21.734812 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f274a3-5df1-446a-8e81-542c52f52155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:21.735850 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f274a3-5df1-446a-8e81-542c52f52155 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:21.736339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f274a3-5df1-446a-8e81-542c52f52155 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:21.736924 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f274a3-5df1-446a-8e81-542c52f52155 None admin] PATH_INFO: `/v3/services` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:21.737825 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f274a3-5df1-446a-8e81-542c52f52155 None admin] RBAC: Adding query filter params (name=image) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:21.738316 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f274a3-5df1-446a-8e81-542c52f52155 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:21.740022 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f274a3-5df1-446a-8e81-542c52f52155 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:21.745954 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 360/726] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:21.747513 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0097fa-daec-4b0e-aa71-db4e3f3ded13 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:21.755223 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f274a3-5df1-446a-8e81-542c52f52155 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:21.796113 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771dec97-c8cc-4347-bd60-8caec8586304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:21.796730 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771dec97-c8cc-4347-bd60-8caec8586304 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:21.797009 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771dec97-c8cc-4347-bd60-8caec8586304 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:21.797281 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771dec97-c8cc-4347-bd60-8caec8586304 None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:21.797915 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771dec97-c8cc-4347-bd60-8caec8586304 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:21.798218 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771dec97-c8cc-4347-bd60-8caec8586304 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:21.799620 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771dec97-c8cc-4347-bd60-8caec8586304 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:21.810705 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 367/727] 10.222.0.30 () {56 vars in 1114 bytes} [Wed Nov 2 07:16:21 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:21.812038 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771dec97-c8cc-4347-bd60-8caec8586304 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:21.873318 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:21.885794 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f274a3-5df1-446a-8e81-542c52f52155 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:21.912926 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:21.926649 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e36b9-4ae6-402d-9c30-10534f0736a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:21.927302 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e36b9-4ae6-402d-9c30-10534f0736a5 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:21.927580 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e36b9-4ae6-402d-9c30-10534f0736a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:21.927846 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e36b9-4ae6-402d-9c30-10534f0736a5 None admin] PATH_INFO: `/v3/services` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:21.928633 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4e36b9-4ae6-402d-9c30-10534f0736a5 None admin] RBAC: Adding query filter params (type=image) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:21.928938 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4e36b9-4ae6-402d-9c30-10534f0736a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:21.931194 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4e36b9-4ae6-402d-9c30-10534f0736a5 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:21.933919 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771dec97-c8cc-4347-bd60-8caec8586304 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:21.936654 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 361/728] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:21 2022] GET /identity/v3/services?type=image => generated 122 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:21.978311 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7551102d-19e8-4b58-b1a2-7ca32b6d6c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:21.978980 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7551102d-19e8-4b58-b1a2-7ca32b6d6c3d None admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:21.979255 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7551102d-19e8-4b58-b1a2-7ca32b6d6c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:21.979525 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7551102d-19e8-4b58-b1a2-7ca32b6d6c3d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:21.980219 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7551102d-19e8-4b58-b1a2-7ca32b6d6c3d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:21.981630 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7551102d-19e8-4b58-b1a2-7ca32b6d6c3d None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:22.008861 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 368/729] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:16:21 2022] POST /identity/v3/endpoints => generated 346 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:16:22.411582 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291fdb1a-d1d1-436b-a9a3-ea5c3c111e4f None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:22.412123 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291fdb1a-d1d1-436b-a9a3-ea5c3c111e4f None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:22.412123 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291fdb1a-d1d1-436b-a9a3-ea5c3c111e4f None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:22.413722 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 362/730] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:22.419325 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995fa14f-9300-4c04-a2a7-84beabdd0de7 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:22.419833 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995fa14f-9300-4c04-a2a7-84beabdd0de7 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:22.419912 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995fa14f-9300-4c04-a2a7-84beabdd0de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:22.438939 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-995fa14f-9300-4c04-a2a7-84beabdd0de7 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:22.440125 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-995fa14f-9300-4c04-a2a7-84beabdd0de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:22.518215 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 369/731] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:22.781499 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291fdb1a-d1d1-436b-a9a3-ea5c3c111e4f None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:22.862967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89131d1-9036-4fec-801d-be012b66b9f4 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:22.863302 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89131d1-9036-4fec-801d-be012b66b9f4 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:22.863545 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89131d1-9036-4fec-801d-be012b66b9f4 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:22.865148 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 370/732] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:22.870409 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27878c63-9101-4202-8478-63af3eaf443a None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:22.870727 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27878c63-9101-4202-8478-63af3eaf443a None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:22.871022 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27878c63-9101-4202-8478-63af3eaf443a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:22.888707 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291fdb1a-d1d1-436b-a9a3-ea5c3c111e4f None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:22.890803 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-27878c63-9101-4202-8478-63af3eaf443a None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:22.892019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-27878c63-9101-4202-8478-63af3eaf443a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:22.929631 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea40711e-b068-40bc-9499-6b2c116f876d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:22.930570 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40711e-b068-40bc-9499-6b2c116f876d None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:22.931061 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40711e-b068-40bc-9499-6b2c116f876d None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:22.931518 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40711e-b068-40bc-9499-6b2c116f876d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:22.932303 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea40711e-b068-40bc-9499-6b2c116f876d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:22.934322 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea40711e-b068-40bc-9499-6b2c116f876d None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:22.945810 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-ea40711e-b068-40bc-9499-6b2c116f876d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:16:22.945810 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:22.945810 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:22.945810 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:22.945810 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:22.945810 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:22.945810 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:22.945810 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:22.945810 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:16:22.945810 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:22.945810 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:22.945810 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:22.945810 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:22.945810 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:22.945810 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:22.945810 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:22.945810 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:22.946833 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:22.946833 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:22.946833 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:22.946833 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:22.946833 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:22.946833 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:22.946833 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:22.946833 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:22.946833 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:22.946833 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:22.946833 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:22.946833 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:22.946833 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:22.946833 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:22.946833 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:22.946833 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:22.947667 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:22.947667 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:22.947667 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:22.947667 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:22.947667 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:22.947667 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:22.947667 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:22.947667 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:22.947667 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:22.947667 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:22.947667 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:22.947667 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:22.947667 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:16:22.947667 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:22.948396 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 363/733] 10.222.0.30 () {56 vars in 1126 bytes} [Wed Nov 2 07:16:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:22.956021 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea40711e-b068-40bc-9499-6b2c116f876d None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:22.961551 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:22.972693 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 371/734] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:16:22.989233 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:23.074268 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea40711e-b068-40bc-9499-6b2c116f876d None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:23.107901 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0bf3f6-b66b-4662-b9df-8ff542a136be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:23.108507 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0bf3f6-b66b-4662-b9df-8ff542a136be None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:23.108782 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0bf3f6-b66b-4662-b9df-8ff542a136be None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:23.109050 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0bf3f6-b66b-4662-b9df-8ff542a136be None admin] PATH_INFO: `/v3/services` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:23.109680 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0bf3f6-b66b-4662-b9df-8ff542a136be None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:23.109975 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0bf3f6-b66b-4662-b9df-8ff542a136be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:23.111377 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0bf3f6-b66b-4662-b9df-8ff542a136be None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:23.116373 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 364/735] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Nov 2 07:16:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:23.122244 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27878c63-9101-4202-8478-63af3eaf443a None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:23.224580 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27878c63-9101-4202-8478-63af3eaf443a None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:23.260073 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4284cea-d164-4e4e-b778-9ab914867c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:23.260696 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4284cea-d164-4e4e-b778-9ab914867c05 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:23.260972 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4284cea-d164-4e4e-b778-9ab914867c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:23.261242 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4284cea-d164-4e4e-b778-9ab914867c05 None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:23.261871 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4284cea-d164-4e4e-b778-9ab914867c05 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:23.262177 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4284cea-d164-4e4e-b778-9ab914867c05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:23.263589 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4284cea-d164-4e4e-b778-9ab914867c05 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:23.266799 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0bf3f6-b66b-4662-b9df-8ff542a136be None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:23.270485 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 372/736] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Nov 2 07:16:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:23.277281 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4284cea-d164-4e4e-b778-9ab914867c05 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:23.350245 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:23.367378 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:23.393795 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0bf3f6-b66b-4662-b9df-8ff542a136be None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:23.398370 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4284cea-d164-4e4e-b778-9ab914867c05 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:23.430588 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0866a9-2f14-45f5-b26e-8c2d06807f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:23.431285 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0866a9-2f14-45f5-b26e-8c2d06807f19 None admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:23.431493 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0866a9-2f14-45f5-b26e-8c2d06807f19 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:23.431760 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0866a9-2f14-45f5-b26e-8c2d06807f19 None admin] PATH_INFO: `/v3/services` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:23.432578 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0866a9-2f14-45f5-b26e-8c2d06807f19 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:23.433774 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0866a9-2f14-45f5-b26e-8c2d06807f19 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:23.441491 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59324c8-f98b-4969-9af2-981a1d5dc488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:23.442826 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59324c8-f98b-4969-9af2-981a1d5dc488 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:23.443286 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59324c8-f98b-4969-9af2-981a1d5dc488 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:23.443739 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59324c8-f98b-4969-9af2-981a1d5dc488 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:23.444618 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59324c8-f98b-4969-9af2-981a1d5dc488 None admin] RBAC: Adding query filter params (interface=admin, service_id=4bf1686c92ef4b4d9b5ea9e67086a1f7, region_id=RegionOne) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:23.445232 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59324c8-f98b-4969-9af2-981a1d5dc488 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:23.446969 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59324c8-f98b-4969-9af2-981a1d5dc488 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:23.459191 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 365/737] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Nov 2 07:16:23 2022] POST /identity/v3/services => generated 242 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:23.459371 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 373/738] 10.222.0.30 () {56 vars in 1249 bytes} [Wed Nov 2 07:16:23 2022] GET /identity/v3/endpoints?service_id=4bf1686c92ef4b4d9b5ea9e67086a1f7&interface=admin®ion_id=RegionOne => generated 193 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:23.460567 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0866a9-2f14-45f5-b26e-8c2d06807f19 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:23.582454 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0866a9-2f14-45f5-b26e-8c2d06807f19 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:23.621727 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8ec71-33c7-4995-89db-96ef9ce0f867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:23.622366 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8ec71-33c7-4995-89db-96ef9ce0f867 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:23.622637 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8ec71-33c7-4995-89db-96ef9ce0f867 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:23.622960 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8ec71-33c7-4995-89db-96ef9ce0f867 None admin] PATH_INFO: `/v3/services` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:23.623576 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd8ec71-33c7-4995-89db-96ef9ce0f867 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:23.625033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd8ec71-33c7-4995-89db-96ef9ce0f867 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:23.632633 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 366/739] 10.222.0.30 () {56 vars in 1087 bytes} [Wed Nov 2 07:16:23 2022] GET /identity/v3/services => generated 1727 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:16:24.190552 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a86854-f16f-44a7-9a78-1ee31609c1ab None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:24.190983 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a86854-f16f-44a7-9a78-1ee31609c1ab None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:24.191155 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a86854-f16f-44a7-9a78-1ee31609c1ab None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:24.192893 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 374/740] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:24.199213 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cfd88c-1c3b-4b31-aae7-3bc8c12f79d1 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:24.199524 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cfd88c-1c3b-4b31-aae7-3bc8c12f79d1 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:24.199764 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cfd88c-1c3b-4b31-aae7-3bc8c12f79d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:24.206696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:24.220682 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-c9cfd88c-1c3b-4b31-aae7-3bc8c12f79d1 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:24.222200 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-c9cfd88c-1c3b-4b31-aae7-3bc8c12f79d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:24.298793 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 367/741] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:24 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:24.301945 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a250d1cf-d355-45b0-ac37-477c2a6f480f None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:24.302323 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a250d1cf-d355-45b0-ac37-477c2a6f480f None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:24.302709 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a250d1cf-d355-45b0-ac37-477c2a6f480f None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:24.304573 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 375/742] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:24.310180 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238aece-5eff-40e3-af9a-203b04d635ab None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:24.310470 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238aece-5eff-40e3-af9a-203b04d635ab None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:24.310771 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238aece-5eff-40e3-af9a-203b04d635ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:24.329527 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-f238aece-5eff-40e3-af9a-203b04d635ab None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:24.331016 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-f238aece-5eff-40e3-af9a-203b04d635ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:24.401999 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 368/743] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:24 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:24.565009 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a250d1cf-d355-45b0-ac37-477c2a6f480f None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:24.672225 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a250d1cf-d355-45b0-ac37-477c2a6f480f None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:24.684727 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f238aece-5eff-40e3-af9a-203b04d635ab None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:24.696683 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:24.708844 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e86d62-b227-4b30-b20d-abb41cbd1959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:24.709454 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e86d62-b227-4b30-b20d-abb41cbd1959 None admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:24.709726 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e86d62-b227-4b30-b20d-abb41cbd1959 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:24.709989 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e86d62-b227-4b30-b20d-abb41cbd1959 None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:24.710795 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e86d62-b227-4b30-b20d-abb41cbd1959 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:24.712280 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e86d62-b227-4b30-b20d-abb41cbd1959 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:24.733325 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 376/744] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Nov 2 07:16:24 2022] POST /identity/v3/services => generated 242 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:24.767360 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:24.813573 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f238aece-5eff-40e3-af9a-203b04d635ab None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:24.859171 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7c6ca-6d8a-45e8-ac6b-b1954b1296fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:24.860048 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7c6ca-6d8a-45e8-ac6b-b1954b1296fd None admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:24.860484 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7c6ca-6d8a-45e8-ac6b-b1954b1296fd None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:24.860914 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7c6ca-6d8a-45e8-ac6b-b1954b1296fd None admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:24.861691 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e7c6ca-6d8a-45e8-ac6b-b1954b1296fd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:24.863307 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e7c6ca-6d8a-45e8-ac6b-b1954b1296fd None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:24.866187 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: WARNING keystone.server.flask.common [None req-65e7c6ca-6d8a-45e8-ac6b-b1954b1296fd None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Nov 02 07:16:24.889097 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.sql.core [None req-65e7c6ca-6d8a-45e8-ac6b-b1954b1296fd None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Nov 02 07:16:24.889097 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 07:16:24.889097 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [parameters: {'id': '0bb03e88c2b841719aebfeaaa9e59480', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Nov 02 07:16:24.889097 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81548) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Nov 02 07:16:24.897913 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-65e7c6ca-6d8a-45e8-ac6b-b1954b1296fd None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0bb03e88c2b841719aebfeaaa9e59480.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0bb03e88c2b841719aebfeaaa9e59480. Nov 02 07:16:24.897913 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:24.897913 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:16:24.897913 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:16:24.897913 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:16:24.897913 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:16:24.897913 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:16:24.897913 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:16:24.897913 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:16:24.897913 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:16:24.897913 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:16:24.897913 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:16:24.897913 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:16:24.899406 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application result.read() Nov 02 07:16:24.899406 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:16:24.899406 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:16:24.899406 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:16:24.899406 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:16:24.899406 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:16:24.899406 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:16:24.899406 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:16:24.899406 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:16:24.899406 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Nov 02 07:16:24.899406 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:24.899406 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Nov 02 07:16:24.899406 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:24.899406 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:24.899406 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Nov 02 07:16:24.899406 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return method(*args, **kwargs) Nov 02 07:16:24.899406 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Nov 02 07:16:24.899406 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return project_ref.to_dict() Nov 02 07:16:24.900909 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:16:24.900909 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:16:24.900909 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Nov 02 07:16:24.900909 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application yield resource Nov 02 07:16:24.900909 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:16:24.900909 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:16:24.900909 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Nov 02 07:16:24.900909 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self.session.rollback() Nov 02 07:16:24.900909 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Nov 02 07:16:24.900909 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self.force_reraise() Nov 02 07:16:24.900909 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Nov 02 07:16:24.900909 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise self.value Nov 02 07:16:24.900909 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Nov 02 07:16:24.900909 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Nov 02 07:16:24.900909 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Nov 02 07:16:24.900909 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application session.commit() Nov 02 07:16:24.900909 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Nov 02 07:16:24.902001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Nov 02 07:16:24.902001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Nov 02 07:16:24.902001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._prepare_impl() Nov 02 07:16:24.902001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Nov 02 07:16:24.902001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self.session.flush() Nov 02 07:16:24.902001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Nov 02 07:16:24.902001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._flush(objects) Nov 02 07:16:24.902001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Nov 02 07:16:24.902001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Nov 02 07:16:24.902001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Nov 02 07:16:24.902001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application compat.raise_( Nov 02 07:16:24.902001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:16:24.902001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception Nov 02 07:16:24.902001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Nov 02 07:16:24.902001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application flush_context.execute() Nov 02 07:16:24.902001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Nov 02 07:16:24.902001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rec.execute(self) Nov 02 07:16:24.902955 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Nov 02 07:16:24.902955 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Nov 02 07:16:24.902955 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Nov 02 07:16:24.902955 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application _emit_insert_statements( Nov 02 07:16:24.902955 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Nov 02 07:16:24.902955 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application c = connection._execute_20( Nov 02 07:16:24.902955 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Nov 02 07:16:24.902955 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Nov 02 07:16:24.902955 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Nov 02 07:16:24.902955 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Nov 02 07:16:24.902955 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Nov 02 07:16:24.902955 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ret = self._execute_context( Nov 02 07:16:24.902955 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Nov 02 07:16:24.902955 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Nov 02 07:16:24.902955 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Nov 02 07:16:24.903724 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Nov 02 07:16:24.903724 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:16:24.903724 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception Nov 02 07:16:24.903724 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:16:24.903724 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:16:24.903724 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:16:24.903724 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:16:24.903724 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:16:24.903724 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:16:24.903724 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:16:24.903724 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:16:24.903724 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:16:24.903724 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:16:24.903724 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:16:24.903724 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application result.read() Nov 02 07:16:24.903724 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:16:24.903724 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:16:24.904607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:16:24.904607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:16:24.904607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:16:24.904607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:16:24.904607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:16:24.904607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:16:24.904607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Nov 02 07:16:24.904607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 07:16:24.904607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [parameters: {'id': '0bb03e88c2b841719aebfeaaa9e59480', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Nov 02 07:16:24.904607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Nov 02 07:16:24.904607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:24.904607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:24.904607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:24.904607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:24.904607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Nov 02 07:16:24.905602 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Nov 02 07:16:24.905602 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Nov 02 07:16:24.905602 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Nov 02 07:16:24.905602 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Nov 02 07:16:24.905602 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:24.905602 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:24.905602 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:24.905602 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:24.905602 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:24.905602 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:24.905602 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:24.905602 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:24.905602 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:24.905602 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:24.905602 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:24.905602 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:24.905602 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:24.906773 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:24.906773 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Nov 02 07:16:24.906773 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Nov 02 07:16:24.906773 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:24.906773 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:24.906773 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Nov 02 07:16:24.906773 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.Conflict( Nov 02 07:16:24.906773 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0bb03e88c2b841719aebfeaaa9e59480. Nov 02 07:16:24.906773 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:24.909329 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 369/745] 10.222.0.30 () {60 vars in 1140 bytes} [Wed Nov 2 07:16:24 2022] POST /identity/v3/projects => generated 262 bytes in 225 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:24.916399 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e86d62-b227-4b30-b20d-abb41cbd1959 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:25.027325 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e86d62-b227-4b30-b20d-abb41cbd1959 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:25.063171 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8603b-be9f-4007-9b80-cf263025841a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:25.064073 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8603b-be9f-4007-9b80-cf263025841a None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:25.064498 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8603b-be9f-4007-9b80-cf263025841a None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:25.064900 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8603b-be9f-4007-9b80-cf263025841a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:25.072562 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d8603b-be9f-4007-9b80-cf263025841a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:25.074289 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d8603b-be9f-4007-9b80-cf263025841a None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:25.086281 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-06d8603b-be9f-4007-9b80-cf263025841a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:16:25.086281 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:25.086281 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:25.086281 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:25.086281 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:25.086281 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:25.086281 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:25.086281 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:25.086281 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:25.086281 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:25.086281 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:25.086281 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:25.086281 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:16:25.086281 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:16:25.086281 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:16:25.087387 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:16:25.087387 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:25.087387 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:25.087387 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:25.087387 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:25.087387 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:25.087387 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:25.087387 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:25.087387 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:25.087387 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:25.087387 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:25.087387 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:25.087387 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:25.087387 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:25.087387 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:25.087387 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:25.087387 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:25.088455 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:16:25.088455 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:16:25.088455 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:16:25.088455 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:16:25.088455 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:16:25.088455 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:16:25.088455 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:16:25.088455 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:25.090498 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 377/746] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:16:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:25.096779 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7c6ca-6d8a-45e8-ac6b-b1954b1296fd None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:25.199872 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7c6ca-6d8a-45e8-ac6b-b1954b1296fd None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:25.214507 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:25.236137 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc22bf-a7b6-4e7e-ab26-19d0d2d677f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:25.237144 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc22bf-a7b6-4e7e-ab26-19d0d2d677f7 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:25.237643 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc22bf-a7b6-4e7e-ab26-19d0d2d677f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:25.238284 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc22bf-a7b6-4e7e-ab26-19d0d2d677f7 None admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:25.238974 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fc22bf-a7b6-4e7e-ab26-19d0d2d677f7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:25.239275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fc22bf-a7b6-4e7e-ab26-19d0d2d677f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:25.240971 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fc22bf-a7b6-4e7e-ab26-19d0d2d677f7 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:25.262255 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 370/747] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Nov 2 07:16:25 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:25.711959 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860a7ae-cbd9-4aa0-92fa-ada2bbae48e9 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:25.712519 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860a7ae-cbd9-4aa0-92fa-ada2bbae48e9 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:25.712519 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860a7ae-cbd9-4aa0-92fa-ada2bbae48e9 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:25.714073 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 378/748] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:25.720321 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91669551-9f20-4c42-bfbf-bf30a37c86d6 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:25.721323 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91669551-9f20-4c42-bfbf-bf30a37c86d6 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:25.721791 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91669551-9f20-4c42-bfbf-bf30a37c86d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:25.742672 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-91669551-9f20-4c42-bfbf-bf30a37c86d6 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:25.744640 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-91669551-9f20-4c42-bfbf-bf30a37c86d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:25.814355 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 371/749] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:25 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:25.837622 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e834e1c-0251-4a8d-972c-3d38bfce24a6 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:25.837921 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e834e1c-0251-4a8d-972c-3d38bfce24a6 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:25.838165 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e834e1c-0251-4a8d-972c-3d38bfce24a6 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:25.839825 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 379/750] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:25.845134 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826c81b-3253-42ca-8183-b9052d32df9d None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:25.845417 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826c81b-3253-42ca-8183-b9052d32df9d None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:25.845680 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826c81b-3253-42ca-8183-b9052d32df9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:25.864083 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-8826c81b-3253-42ca-8183-b9052d32df9d None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:25.865177 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-8826c81b-3253-42ca-8183-b9052d32df9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:25.932743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 372/751] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:25 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:26.088774 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e834e1c-0251-4a8d-972c-3d38bfce24a6 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:26.136898 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:26.194981 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e834e1c-0251-4a8d-972c-3d38bfce24a6 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:26.203385 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8826c81b-3253-42ca-8183-b9052d32df9d None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:26.240962 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd2ceda-a433-4ecc-88e5-f6c45776572c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:26.241661 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd2ceda-a433-4ecc-88e5-f6c45776572c None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:26.241943 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd2ceda-a433-4ecc-88e5-f6c45776572c None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:26.242287 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd2ceda-a433-4ecc-88e5-f6c45776572c None admin] PATH_INFO: `/v3/services/compute` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:26.242903 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd2ceda-a433-4ecc-88e5-f6c45776572c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:26.244492 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:26.244673 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd2ceda-a433-4ecc-88e5-f6c45776572c None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:26.255653 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-1dd2ceda-a433-4ecc-88e5-f6c45776572c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:16:26.255653 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:26.255653 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:26.255653 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:26.255653 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:26.255653 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:26.255653 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:26.255653 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:26.255653 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:16:26.255653 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:26.255653 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:26.255653 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:26.255653 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:26.255653 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:26.255653 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:26.255653 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:26.255653 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:26.256659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:26.256659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:26.256659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:26.256659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:26.256659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:26.256659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:26.256659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:26.256659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:26.256659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:26.256659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:26.256659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:26.256659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:26.256659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:26.256659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:26.256659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:26.256659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:26.258020 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:26.258020 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:26.258020 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:26.258020 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:26.258020 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:26.258020 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:26.258020 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:26.258020 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:26.258020 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:26.258020 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:26.258020 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:26.258020 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:26.258020 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:16:26.258020 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:26.258020 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 380/752] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:16:26 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:26.263275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd2ceda-a433-4ecc-88e5-f6c45776572c None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:26.327936 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8826c81b-3253-42ca-8183-b9052d32df9d None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:26.365482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c48c5eb-2adb-46ae-83f1-701ea87d5016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:26.366602 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48c5eb-2adb-46ae-83f1-701ea87d5016 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:26.367276 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48c5eb-2adb-46ae-83f1-701ea87d5016 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:26.367737 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48c5eb-2adb-46ae-83f1-701ea87d5016 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:26.368661 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c48c5eb-2adb-46ae-83f1-701ea87d5016 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:26.370843 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c48c5eb-2adb-46ae-83f1-701ea87d5016 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:26.372106 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd2ceda-a433-4ecc-88e5-f6c45776572c None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:26.380162 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-6c48c5eb-2adb-46ae-83f1-701ea87d5016 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:16:26.380162 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:26.380162 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:26.380162 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:26.380162 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:26.380162 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:26.380162 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:26.380162 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:26.380162 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:16:26.380162 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:26.380162 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:26.380162 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:26.380162 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:26.380162 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:26.380162 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:26.380162 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:26.380162 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:26.381130 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:26.381130 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:26.381130 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:26.381130 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:26.381130 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:26.381130 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:26.381130 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:26.381130 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:26.381130 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:26.381130 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:26.381130 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:26.381130 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:26.381130 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:26.381130 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:26.381130 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:26.381130 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:26.382020 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:26.382020 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:26.382020 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:26.382020 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:26.382020 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:26.382020 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:26.382020 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:26.382020 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:26.382020 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:26.382020 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:26.382020 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:26.382020 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:26.382020 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 07:16:26.382020 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:26.383765 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:26.385535 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 373/753] 10.222.0.30 () {56 vars in 1126 bytes} [Wed Nov 2 07:16:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:26.393482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c48c5eb-2adb-46ae-83f1-701ea87d5016 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:26.419680 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71408ee-c98c-4b11-8266-9a7f386eb2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:26.420380 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71408ee-c98c-4b11-8266-9a7f386eb2b3 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:26.420901 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71408ee-c98c-4b11-8266-9a7f386eb2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:26.421176 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71408ee-c98c-4b11-8266-9a7f386eb2b3 None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:26.422207 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71408ee-c98c-4b11-8266-9a7f386eb2b3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:26.422984 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71408ee-c98c-4b11-8266-9a7f386eb2b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:26.424651 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71408ee-c98c-4b11-8266-9a7f386eb2b3 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:26.434742 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 381/754] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Nov 2 07:16:26 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:26.437458 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71408ee-c98c-4b11-8266-9a7f386eb2b3 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:26.496815 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:26.500822 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c48c5eb-2adb-46ae-83f1-701ea87d5016 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:26.543488 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369bbd7-1d4b-4522-b232-8411902208ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:26.544657 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369bbd7-1d4b-4522-b232-8411902208ae None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:26.545593 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369bbd7-1d4b-4522-b232-8411902208ae None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:26.546208 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369bbd7-1d4b-4522-b232-8411902208ae None admin] PATH_INFO: `/v3/services` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:26.547105 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9369bbd7-1d4b-4522-b232-8411902208ae None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:26.547778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9369bbd7-1d4b-4522-b232-8411902208ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:26.549682 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9369bbd7-1d4b-4522-b232-8411902208ae None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:26.557580 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 374/755] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Nov 2 07:16:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:26.563205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369bbd7-1d4b-4522-b232-8411902208ae None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:26.568946 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71408ee-c98c-4b11-8266-9a7f386eb2b3 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:26.621830 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37432ae2-4272-4c53-8255-0ff211464962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:26.623255 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37432ae2-4272-4c53-8255-0ff211464962 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:26.624185 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37432ae2-4272-4c53-8255-0ff211464962 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:26.624961 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37432ae2-4272-4c53-8255-0ff211464962 None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:26.625857 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37432ae2-4272-4c53-8255-0ff211464962 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:26.626606 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37432ae2-4272-4c53-8255-0ff211464962 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:26.629043 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37432ae2-4272-4c53-8255-0ff211464962 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:26.642391 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 382/756] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Nov 2 07:16:26 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:26.643834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37432ae2-4272-4c53-8255-0ff211464962 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:26.686469 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369bbd7-1d4b-4522-b232-8411902208ae None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:26.690344 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:26.723067 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf7d2e0-66b1-43bf-b240-cf6d2f8548c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:26.723660 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf7d2e0-66b1-43bf-b240-cf6d2f8548c6 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:26.723933 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf7d2e0-66b1-43bf-b240-cf6d2f8548c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:26.724197 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf7d2e0-66b1-43bf-b240-cf6d2f8548c6 None admin] PATH_INFO: `/v3/services` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:26.724818 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf7d2e0-66b1-43bf-b240-cf6d2f8548c6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:26.725193 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf7d2e0-66b1-43bf-b240-cf6d2f8548c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:26.727033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf7d2e0-66b1-43bf-b240-cf6d2f8548c6 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:26.737226 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 375/757] 10.222.0.30 () {56 vars in 1122 bytes} [Wed Nov 2 07:16:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:26.738082 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf7d2e0-66b1-43bf-b240-cf6d2f8548c6 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:26.767177 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:26.772545 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37432ae2-4272-4c53-8255-0ff211464962 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:26.810786 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b2b10-3382-4f4f-ae78-c780729374e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:26.811441 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b2b10-3382-4f4f-ae78-c780729374e8 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:26.811809 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b2b10-3382-4f4f-ae78-c780729374e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:26.812160 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b2b10-3382-4f4f-ae78-c780729374e8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:26.812945 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467b2b10-3382-4f4f-ae78-c780729374e8 None admin] RBAC: Adding query filter params (interface=public, service_id=2d5df59d038a474fb061fe7d20f1feb1, region_id=RegionOne) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:26.813345 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467b2b10-3382-4f4f-ae78-c780729374e8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:26.814949 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467b2b10-3382-4f4f-ae78-c780729374e8 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:26.826163 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 383/758] 10.222.0.30 () {56 vars in 1251 bytes} [Wed Nov 2 07:16:26 2022] GET /identity/v3/endpoints?service_id=2d5df59d038a474fb061fe7d20f1feb1&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:26.827037 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b2b10-3382-4f4f-ae78-c780729374e8 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:26.855458 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf7d2e0-66b1-43bf-b240-cf6d2f8548c6 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:26.892800 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ce171-c919-444a-8890-0ba01de79dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:26.893632 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ce171-c919-444a-8890-0ba01de79dc5 None admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:26.894054 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ce171-c919-444a-8890-0ba01de79dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:26.894486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ce171-c919-444a-8890-0ba01de79dc5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:26.895337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ce171-c919-444a-8890-0ba01de79dc5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:26.897129 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ce171-c919-444a-8890-0ba01de79dc5 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:26.920278 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 376/759] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:16:26 2022] POST /identity/v3/endpoints => generated 325 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:26.946142 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:26.949889 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b2b10-3382-4f4f-ae78-c780729374e8 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:26.997622 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c42fe-1ecc-42f3-8904-0c1209bef8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:26.998585 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c42fe-1ecc-42f3-8904-0c1209bef8e2 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:26.999172 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c42fe-1ecc-42f3-8904-0c1209bef8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:26.999618 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c42fe-1ecc-42f3-8904-0c1209bef8e2 None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:27.000472 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c42fe-1ecc-42f3-8904-0c1209bef8e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:27.002063 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c42fe-1ecc-42f3-8904-0c1209bef8e2 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:27.010403 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 384/760] 10.222.0.30 () {56 vars in 1087 bytes} [Wed Nov 2 07:16:26 2022] GET /identity/v3/services => generated 1957 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:16:27.015319 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b8780-0309-4bb0-96ad-caccb585ce51 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:27.015702 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b8780-0309-4bb0-96ad-caccb585ce51 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:27.016055 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b8780-0309-4bb0-96ad-caccb585ce51 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:27.017671 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 377/761] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:16:27.026712 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4daf57e-3345-4275-a9eb-accb57dd0e20 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:27.026712 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4daf57e-3345-4275-a9eb-accb57dd0e20 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:27.026712 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4daf57e-3345-4275-a9eb-accb57dd0e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:27.045798 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-b4daf57e-3345-4275-a9eb-accb57dd0e20 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:27.047202 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-b4daf57e-3345-4275-a9eb-accb57dd0e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:27.133891 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 385/762] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:27 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:27.407972 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0b8780-0309-4bb0-96ad-caccb585ce51 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:27.505238 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:27.527205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0b8780-0309-4bb0-96ad-caccb585ce51 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:27.562158 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e34394a-bf4d-4fc8-a039-8c81058548b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:27.563220 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34394a-bf4d-4fc8-a039-8c81058548b8 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:27.563740 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34394a-bf4d-4fc8-a039-8c81058548b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:27.564231 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34394a-bf4d-4fc8-a039-8c81058548b8 None admin] PATH_INFO: `/v3/services/image` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:27.565026 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e34394a-bf4d-4fc8-a039-8c81058548b8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:27.566603 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e34394a-bf4d-4fc8-a039-8c81058548b8 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:27.576743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-1e34394a-bf4d-4fc8-a039-8c81058548b8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:16:27.576743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:27.576743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:27.576743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:27.576743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:27.576743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:27.576743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:27.576743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:27.576743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:16:27.576743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:27.576743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:27.576743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:27.576743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:27.576743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:27.576743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:27.576743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:27.576743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:27.577821 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:27.577821 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:27.577821 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:27.577821 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:27.577821 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:27.577821 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:27.577821 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:27.577821 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:27.577821 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:27.577821 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:27.577821 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:27.577821 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:27.577821 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:27.577821 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:27.577821 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:27.577821 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:27.578719 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:27.578719 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:27.578719 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:27.578719 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:27.578719 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:27.578719 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:27.578719 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:27.578719 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:27.578719 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:27.578719 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:27.578719 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:27.578719 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:27.578719 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:16:27.578719 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:27.581428 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 378/763] 10.222.0.30 () {56 vars in 1105 bytes} [Wed Nov 2 07:16:27 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:27.587664 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4daf57e-3345-4275-a9eb-accb57dd0e20 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:27.592091 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a367180-3d70-42f0-a8dc-08ec4951b829 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:27.592371 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a367180-3d70-42f0-a8dc-08ec4951b829 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:27.592642 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a367180-3d70-42f0-a8dc-08ec4951b829 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:27.594256 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 379/764] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:16:27.599461 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005549af-6574-48dc-8c4c-46cad9304aff None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:27.599737 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005549af-6574-48dc-8c4c-46cad9304aff None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:27.600003 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005549af-6574-48dc-8c4c-46cad9304aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:27.619252 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-005549af-6574-48dc-8c4c-46cad9304aff None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:27.620501 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-005549af-6574-48dc-8c4c-46cad9304aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:27.698558 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 380/765] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:27 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:16:27.710192 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:27.721304 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4daf57e-3345-4275-a9eb-accb57dd0e20 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:27.762284 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51fd4fc-72b3-45ee-9251-fb69f295a550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:27.762945 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51fd4fc-72b3-45ee-9251-fb69f295a550 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:27.763223 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51fd4fc-72b3-45ee-9251-fb69f295a550 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:27.763494 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51fd4fc-72b3-45ee-9251-fb69f295a550 None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:27.764123 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51fd4fc-72b3-45ee-9251-fb69f295a550 None admin] RBAC: Adding query filter params (name=image) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:27.764421 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51fd4fc-72b3-45ee-9251-fb69f295a550 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:27.765790 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51fd4fc-72b3-45ee-9251-fb69f295a550 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:27.771325 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 386/766] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:27.777627 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005549af-6574-48dc-8c4c-46cad9304aff None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:27.858326 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:27.877625 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005549af-6574-48dc-8c4c-46cad9304aff None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:27.917454 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1b777-11c9-4d18-9b12-611e8166051d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:27.918085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1b777-11c9-4d18-9b12-611e8166051d None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:27.918364 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1b777-11c9-4d18-9b12-611e8166051d None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:27.918632 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1b777-11c9-4d18-9b12-611e8166051d None admin] PATH_INFO: `/v3/services` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:27.919310 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1b777-11c9-4d18-9b12-611e8166051d None admin] RBAC: Adding query filter params (type=image) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:27.919700 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1b777-11c9-4d18-9b12-611e8166051d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:27.921131 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1b777-11c9-4d18-9b12-611e8166051d None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:27.927314 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 381/767] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:27.933490 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51fd4fc-72b3-45ee-9251-fb69f295a550 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:27.980501 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1b777-11c9-4d18-9b12-611e8166051d None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:28.057947 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51fd4fc-72b3-45ee-9251-fb69f295a550 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:28.087327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1b777-11c9-4d18-9b12-611e8166051d None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:28.098672 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a209f-96f0-4dc0-afb8-e06cfa306e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:28.099556 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a209f-96f0-4dc0-afb8-e06cfa306e9c None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:28.099832 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a209f-96f0-4dc0-afb8-e06cfa306e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:28.100097 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a209f-96f0-4dc0-afb8-e06cfa306e9c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:28.101312 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095a209f-96f0-4dc0-afb8-e06cfa306e9c None admin] RBAC: Adding query filter params (interface=public, service_id=1adfbffadaa74469a698963c4b900aef, region_id=RegionOne) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:28.101628 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095a209f-96f0-4dc0-afb8-e06cfa306e9c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:28.103683 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095a209f-96f0-4dc0-afb8-e06cfa306e9c None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:28.110305 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 387/768] 10.222.0.30 () {56 vars in 1251 bytes} [Wed Nov 2 07:16:27 2022] GET /identity/v3/endpoints?service_id=1adfbffadaa74469a698963c4b900aef&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:28.115645 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a209f-96f0-4dc0-afb8-e06cfa306e9c None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:28.126039 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909cecbb-dfe8-4812-a6dd-379194089a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:28.126782 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909cecbb-dfe8-4812-a6dd-379194089a6d None admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:28.127057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909cecbb-dfe8-4812-a6dd-379194089a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:28.127320 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909cecbb-dfe8-4812-a6dd-379194089a6d None admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:28.127962 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909cecbb-dfe8-4812-a6dd-379194089a6d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:28.129491 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909cecbb-dfe8-4812-a6dd-379194089a6d None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:28.136070 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:28.146945 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.sql.core [None req-909cecbb-dfe8-4812-a6dd-379194089a6d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Nov 02 07:16:28.146945 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Nov 02 07:16:28.146945 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [parameters: {'user_id': '12adbaedfc35461ea593b871b77cdc5b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Nov 02 07:16:28.146945 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81548) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Nov 02 07:16:28.155750 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-909cecbb-dfe8-4812-a6dd-379194089a6d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Nov 02 07:16:28.155750 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:28.155750 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:16:28.155750 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:16:28.155750 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:16:28.155750 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:16:28.155750 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:16:28.155750 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:16:28.155750 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:16:28.155750 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:16:28.155750 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:16:28.155750 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:16:28.155750 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:16:28.155750 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application result.read() Nov 02 07:16:28.155750 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:16:28.156633 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:16:28.156633 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:16:28.156633 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:16:28.156633 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:16:28.156633 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:16:28.156633 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:16:28.156633 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:16:28.156633 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Nov 02 07:16:28.156633 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:28.156633 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Nov 02 07:16:28.156633 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:28.156633 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:28.156633 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Nov 02 07:16:28.156633 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return method(*args, **kwargs) Nov 02 07:16:28.156633 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Nov 02 07:16:28.156633 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Nov 02 07:16:28.156633 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:16:28.156633 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:16:28.157548 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Nov 02 07:16:28.157548 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application yield resource Nov 02 07:16:28.157548 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:16:28.157548 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:16:28.157548 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Nov 02 07:16:28.157548 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self.session.rollback() Nov 02 07:16:28.157548 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Nov 02 07:16:28.157548 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self.force_reraise() Nov 02 07:16:28.157548 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Nov 02 07:16:28.157548 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise self.value Nov 02 07:16:28.157548 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Nov 02 07:16:28.157548 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Nov 02 07:16:28.157548 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Nov 02 07:16:28.157548 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application session.commit() Nov 02 07:16:28.157548 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Nov 02 07:16:28.157548 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Nov 02 07:16:28.157548 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Nov 02 07:16:28.158422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._prepare_impl() Nov 02 07:16:28.158422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Nov 02 07:16:28.158422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self.session.flush() Nov 02 07:16:28.158422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Nov 02 07:16:28.158422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._flush(objects) Nov 02 07:16:28.158422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Nov 02 07:16:28.158422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Nov 02 07:16:28.158422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Nov 02 07:16:28.158422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application compat.raise_( Nov 02 07:16:28.158422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:16:28.158422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception Nov 02 07:16:28.158422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Nov 02 07:16:28.158422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application flush_context.execute() Nov 02 07:16:28.158422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Nov 02 07:16:28.158422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rec.execute(self) Nov 02 07:16:28.158422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Nov 02 07:16:28.158422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Nov 02 07:16:28.159414 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Nov 02 07:16:28.159414 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application _emit_insert_statements( Nov 02 07:16:28.159414 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Nov 02 07:16:28.159414 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application result = connection._execute_20( Nov 02 07:16:28.159414 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Nov 02 07:16:28.159414 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Nov 02 07:16:28.159414 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Nov 02 07:16:28.159414 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Nov 02 07:16:28.159414 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Nov 02 07:16:28.159414 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ret = self._execute_context( Nov 02 07:16:28.159414 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Nov 02 07:16:28.159414 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Nov 02 07:16:28.159414 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Nov 02 07:16:28.159414 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Nov 02 07:16:28.159414 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:16:28.159414 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception Nov 02 07:16:28.160232 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:16:28.160232 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:16:28.160232 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:16:28.160232 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:16:28.160232 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:16:28.160232 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:16:28.160232 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:16:28.160232 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:16:28.160232 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:16:28.160232 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:16:28.160232 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:16:28.160232 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application result.read() Nov 02 07:16:28.160232 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:16:28.160232 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:16:28.160232 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:16:28.160232 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:16:28.161060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:16:28.161060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:16:28.161060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:16:28.161060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:16:28.161060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Nov 02 07:16:28.161060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Nov 02 07:16:28.161060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [parameters: {'user_id': '12adbaedfc35461ea593b871b77cdc5b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Nov 02 07:16:28.161060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Nov 02 07:16:28.161060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:28.161060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:28.161060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:28.161060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:28.161060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:28.161060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:28.161060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:28.161836 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:28.161836 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:28.161836 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:28.161836 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:28.161836 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:28.161836 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:28.161836 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:28.161836 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Nov 02 07:16:28.161836 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Nov 02 07:16:28.161836 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:28.161836 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:28.161836 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:16:28.161836 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:16:28.161836 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:16:28.161836 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:16:28.161836 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Nov 02 07:16:28.161836 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Nov 02 07:16:28.162792 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Nov 02 07:16:28.162792 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Nov 02 07:16:28.162792 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Nov 02 07:16:28.162792 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Nov 02 07:16:28.162792 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Nov 02 07:16:28.162792 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:28.162792 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 382/769] 10.222.0.30 () {60 vars in 1131 bytes} [Wed Nov 2 07:16:27 2022] POST /identity/v3/users => generated 160 bytes in 177 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:28.163480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909cecbb-dfe8-4812-a6dd-379194089a6d None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:28.173072 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:28.230789 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a209f-96f0-4dc0-afb8-e06cfa306e9c None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:28.265544 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae470ea8-ea5b-4424-b216-95ec958caf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:28.266455 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae470ea8-ea5b-4424-b216-95ec958caf6b None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:28.266926 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae470ea8-ea5b-4424-b216-95ec958caf6b None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:28.267394 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae470ea8-ea5b-4424-b216-95ec958caf6b None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:28.268165 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae470ea8-ea5b-4424-b216-95ec958caf6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:28.269706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae470ea8-ea5b-4424-b216-95ec958caf6b None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:28.286688 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 388/770] 10.222.0.30 () {56 vars in 1087 bytes} [Wed Nov 2 07:16:28 2022] GET /identity/v3/services => generated 1957 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:16:28.287123 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909cecbb-dfe8-4812-a6dd-379194089a6d None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:28.331083 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bcdf4c-ff64-4724-8fbf-919793d26e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:28.331725 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bcdf4c-ff64-4724-8fbf-919793d26e70 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:28.331996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bcdf4c-ff64-4724-8fbf-919793d26e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:28.332266 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bcdf4c-ff64-4724-8fbf-919793d26e70 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:28.341838 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bcdf4c-ff64-4724-8fbf-919793d26e70 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:28.343545 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bcdf4c-ff64-4724-8fbf-919793d26e70 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:28.356853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-85bcdf4c-ff64-4724-8fbf-919793d26e70 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 07:16:28.356853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:28.356853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:28.356853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:28.356853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:28.356853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:28.356853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:28.356853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:28.356853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:28.356853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:28.356853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:28.356853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:28.356853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:16:28.356853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:16:28.356853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:16:28.356853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:16:28.357865 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:28.357865 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:28.357865 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:16:28.357865 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:16:28.357865 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:16:28.357865 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:16:28.357865 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:28.357865 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:28.357865 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:28.357865 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:28.357865 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:28.357865 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:28.357865 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:28.357865 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:28.357865 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:28.357865 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:28.357865 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:28.357865 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:28.358832 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:28.358832 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:28.358832 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:16:28.358832 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:16:28.358832 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:16:28.358832 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:16:28.358832 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:16:28.358832 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:16:28.358832 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 07:16:28.358832 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:28.358832 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 383/771] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:28.364056 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae470ea8-ea5b-4424-b216-95ec958caf6b None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:28.457933 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:28.477758 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae470ea8-ea5b-4424-b216-95ec958caf6b None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:28.512662 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33762c5-e654-4709-81a9-c3fbc3316b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:28.513236 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33762c5-e654-4709-81a9-c3fbc3316b13 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:28.513512 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33762c5-e654-4709-81a9-c3fbc3316b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:28.513779 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33762c5-e654-4709-81a9-c3fbc3316b13 None admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:28.514471 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33762c5-e654-4709-81a9-c3fbc3316b13 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:28.514795 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33762c5-e654-4709-81a9-c3fbc3316b13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:28.516284 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33762c5-e654-4709-81a9-c3fbc3316b13 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:28.552447 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 389/772] 10.222.0.30 () {56 vars in 1101 bytes} [Wed Nov 2 07:16:28 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:29.317278 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a4d6f5-701b-4198-a665-da544ad440ab None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:29.317843 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a4d6f5-701b-4198-a665-da544ad440ab None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:29.317843 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a4d6f5-701b-4198-a665-da544ad440ab None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:29.319793 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222402be-fac3-4844-b5e3-7c9edb541fbd None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:29.320233 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222402be-fac3-4844-b5e3-7c9edb541fbd None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:29.320312 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222402be-fac3-4844-b5e3-7c9edb541fbd None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:29.321735 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 390/773] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:16:29.322294 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 384/774] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:29.327189 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57cf4c-7e9b-494b-a540-baa7f2286d89 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:29.327435 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57cf4c-7e9b-494b-a540-baa7f2286d89 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:29.327533 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb9aa8-a607-4451-bae3-dc8941e8d90c None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:29.327717 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57cf4c-7e9b-494b-a540-baa7f2286d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:29.327814 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb9aa8-a607-4451-bae3-dc8941e8d90c None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:29.327975 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb9aa8-a607-4451-bae3-dc8941e8d90c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:29.346349 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-ef57cf4c-7e9b-494b-a540-baa7f2286d89 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:29.347599 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-ef57cf4c-7e9b-494b-a540-baa7f2286d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:29.349997 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-adeb9aa8-a607-4451-bae3-dc8941e8d90c None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:29.351812 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-adeb9aa8-a607-4451-bae3-dc8941e8d90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:29.417674 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:29.424073 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 391/775] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:29 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:29.430998 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 385/776] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:29 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:16:29.687734 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57cf4c-7e9b-494b-a540-baa7f2286d89 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:29.710303 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb9aa8-a607-4451-bae3-dc8941e8d90c None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:29.765834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:29.798952 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57cf4c-7e9b-494b-a540-baa7f2286d89 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:29.823497 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb9aa8-a607-4451-bae3-dc8941e8d90c None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:29.834905 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517b928-02d7-4aab-b1c2-792a4ac68847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:29.835758 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517b928-02d7-4aab-b1c2-792a4ac68847 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:29.835891 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517b928-02d7-4aab-b1c2-792a4ac68847 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:29.836070 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517b928-02d7-4aab-b1c2-792a4ac68847 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:29.836625 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8517b928-02d7-4aab-b1c2-792a4ac68847 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:29.838006 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8517b928-02d7-4aab-b1c2-792a4ac68847 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:29.848011 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-8517b928-02d7-4aab-b1c2-792a4ac68847 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:16:29.848011 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:29.848011 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:29.848011 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:29.848011 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:29.848011 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:29.848011 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:29.848011 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:29.848011 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:16:29.848011 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:29.848011 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:29.848011 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:29.848011 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:29.848011 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:29.848011 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:29.848011 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:29.848011 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:29.849186 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:29.849186 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:29.849186 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:29.849186 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:29.849186 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:29.849186 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:29.849186 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:29.849186 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:29.849186 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:29.849186 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:29.849186 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:29.849186 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:29.849186 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:29.849186 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:29.849186 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:29.849186 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:29.850010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:29.850010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:29.850010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:29.850010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:29.850010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:29.850010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:29.850010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:29.850010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:29.850010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:29.850010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:29.850010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:29.850010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:29.850010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 07:16:29.850010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:29.850010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 392/777] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:16:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:29.855908 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517b928-02d7-4aab-b1c2-792a4ac68847 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:29.864794 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb64639c-e874-4823-a57b-38cf25d6e4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:29.865875 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb64639c-e874-4823-a57b-38cf25d6e4b6 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:29.866297 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb64639c-e874-4823-a57b-38cf25d6e4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:29.866810 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb64639c-e874-4823-a57b-38cf25d6e4b6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:29.868312 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb64639c-e874-4823-a57b-38cf25d6e4b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:29.870120 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb64639c-e874-4823-a57b-38cf25d6e4b6 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:29.872055 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 386/778] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:29.877529 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb64639c-e874-4823-a57b-38cf25d6e4b6 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:29.942583 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:29.972087 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517b928-02d7-4aab-b1c2-792a4ac68847 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:29.983745 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb64639c-e874-4823-a57b-38cf25d6e4b6 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:30.007736 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a66cbb3-9def-4719-97a5-85c5387d0048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:30.008321 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a66cbb3-9def-4719-97a5-85c5387d0048 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:30.008671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a66cbb3-9def-4719-97a5-85c5387d0048 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:30.008939 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a66cbb3-9def-4719-97a5-85c5387d0048 None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:30.009562 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a66cbb3-9def-4719-97a5-85c5387d0048 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:30.009863 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a66cbb3-9def-4719-97a5-85c5387d0048 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:30.011366 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a66cbb3-9def-4719-97a5-85c5387d0048 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:30.012756 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:30.021908 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659528d-3217-4ad1-9607-15a109e82b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:30.021908 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659528d-3217-4ad1-9607-15a109e82b5d None admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:30.021908 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659528d-3217-4ad1-9607-15a109e82b5d None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:30.021908 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659528d-3217-4ad1-9607-15a109e82b5d None admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:30.022391 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1659528d-3217-4ad1-9607-15a109e82b5d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:30.023301 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 393/779] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Nov 2 07:16:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:30.023410 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1659528d-3217-4ad1-9607-15a109e82b5d None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:30.024151 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a66cbb3-9def-4719-97a5-85c5387d0048 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:30.053736 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 387/780] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Nov 2 07:16:29 2022] POST /identity/v3/projects => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:16:30.153775 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a66cbb3-9def-4719-97a5-85c5387d0048 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:30.192362 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01d921-d438-49fa-a7e9-53bf61163b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:30.192984 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01d921-d438-49fa-a7e9-53bf61163b2c None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:30.193254 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01d921-d438-49fa-a7e9-53bf61163b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:30.193514 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01d921-d438-49fa-a7e9-53bf61163b2c None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:30.194211 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef01d921-d438-49fa-a7e9-53bf61163b2c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:30.194649 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef01d921-d438-49fa-a7e9-53bf61163b2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:30.196385 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef01d921-d438-49fa-a7e9-53bf61163b2c None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:30.202400 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 394/781] 10.222.0.30 () {56 vars in 1112 bytes} [Wed Nov 2 07:16:30 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:30.208162 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659528d-3217-4ad1-9607-15a109e82b5d None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:30.313858 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659528d-3217-4ad1-9607-15a109e82b5d None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:30.330641 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:30.351605 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a563b21f-b225-41df-8c4c-d6283d193b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:30.352254 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563b21f-b225-41df-8c4c-d6283d193b05 None admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:30.352526 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563b21f-b225-41df-8c4c-d6283d193b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:30.352793 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563b21f-b225-41df-8c4c-d6283d193b05 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:30.353497 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a563b21f-b225-41df-8c4c-d6283d193b05 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:30.354972 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a563b21f-b225-41df-8c4c-d6283d193b05 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:30.380935 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 388/782] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:16:30 2022] POST /identity/v3/endpoints => generated 334 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:30.472665 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df4446-5a43-45cd-bae4-285baa8bac7d None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:30.473196 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df4446-5a43-45cd-bae4-285baa8bac7d None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:30.473196 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df4446-5a43-45cd-bae4-285baa8bac7d None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:30.474511 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 395/783] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:30.480872 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534307bb-6e21-4fd2-933b-6a19799102ff None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:30.481206 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534307bb-6e21-4fd2-933b-6a19799102ff None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:30.481370 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534307bb-6e21-4fd2-933b-6a19799102ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:30.500649 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-534307bb-6e21-4fd2-933b-6a19799102ff None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:30.501877 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-534307bb-6e21-4fd2-933b-6a19799102ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:30.587137 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 389/784] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:30 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:30.848962 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df4446-5a43-45cd-bae4-285baa8bac7d None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:30.914870 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:30.958020 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df4446-5a43-45cd-bae4-285baa8bac7d None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:31.114123 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f933d825-0e68-4b46-bab7-db18e142e5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:31.114771 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933d825-0e68-4b46-bab7-db18e142e5fe None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:31.115041 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933d825-0e68-4b46-bab7-db18e142e5fe None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:31.115306 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933d825-0e68-4b46-bab7-db18e142e5fe None admin] PATH_INFO: `/v3/services/image` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:31.115978 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f933d825-0e68-4b46-bab7-db18e142e5fe None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:31.117345 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f933d825-0e68-4b46-bab7-db18e142e5fe None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:31.126159 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-f933d825-0e68-4b46-bab7-db18e142e5fe None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:16:31.126159 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:31.126159 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:16:31.126159 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:16:31.126159 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:16:31.126159 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:16:31.126159 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:16:31.126159 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:31.126159 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:16:31.126159 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:31.126159 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:31.126159 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:31.126159 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:31.126159 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:31.126159 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:31.126159 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:31.126159 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:31.127241 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:31.127241 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:31.127241 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:31.127241 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:31.127241 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:31.127241 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:31.127241 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:16:31.127241 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:16:31.127241 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:16:31.127241 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:16:31.127241 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:31.127241 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:31.127241 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:31.127241 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:31.127241 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:31.127241 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:31.128095 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:31.128095 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:31.128095 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:31.128095 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:31.128095 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:31.128095 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:31.128095 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:31.128095 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:31.128095 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:31.128095 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:31.128095 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:16:31.128095 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:16:31.128095 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 07:16:31.128095 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:31.128095 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 396/785] 10.222.0.30 () {56 vars in 1105 bytes} [Wed Nov 2 07:16:30 2022] GET /identity/v3/services/image => generated 86 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:31.133560 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534307bb-6e21-4fd2-933b-6a19799102ff None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:31.233265 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:31.237572 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534307bb-6e21-4fd2-933b-6a19799102ff None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:31.273928 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204d390-8ac6-49fd-a462-9e40bd2cdfc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:31.274904 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204d390-8ac6-49fd-a462-9e40bd2cdfc0 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:31.275181 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204d390-8ac6-49fd-a462-9e40bd2cdfc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:31.275448 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204d390-8ac6-49fd-a462-9e40bd2cdfc0 None admin] PATH_INFO: `/v3/services` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:31.276081 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6204d390-8ac6-49fd-a462-9e40bd2cdfc0 None admin] RBAC: Adding query filter params (name=image) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:31.276380 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6204d390-8ac6-49fd-a462-9e40bd2cdfc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:31.277754 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6204d390-8ac6-49fd-a462-9e40bd2cdfc0 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:31.282910 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 390/786] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:31.288993 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f933d825-0e68-4b46-bab7-db18e142e5fe None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:31.392855 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f933d825-0e68-4b46-bab7-db18e142e5fe None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:31.418779 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:31.429383 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ebc3d3-4022-45f4-9feb-cba39bcf0859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:31.429988 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ebc3d3-4022-45f4-9feb-cba39bcf0859 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:31.430255 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ebc3d3-4022-45f4-9feb-cba39bcf0859 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:31.430511 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ebc3d3-4022-45f4-9feb-cba39bcf0859 None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:31.431167 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ebc3d3-4022-45f4-9feb-cba39bcf0859 None admin] RBAC: Adding query filter params (type=image) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:31.431443 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ebc3d3-4022-45f4-9feb-cba39bcf0859 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:31.432805 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ebc3d3-4022-45f4-9feb-cba39bcf0859 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:31.437716 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 397/787] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:31 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:31.443364 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204d390-8ac6-49fd-a462-9e40bd2cdfc0 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:31.542594 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204d390-8ac6-49fd-a462-9e40bd2cdfc0 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:31.575828 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97929d-4ac8-414c-9b30-4c45614e3e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:31.576424 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97929d-4ac8-414c-9b30-4c45614e3e13 None admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:31.576693 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97929d-4ac8-414c-9b30-4c45614e3e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:31.576957 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97929d-4ac8-414c-9b30-4c45614e3e13 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:31.577613 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf97929d-4ac8-414c-9b30-4c45614e3e13 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:31.579010 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf97929d-4ac8-414c-9b30-4c45614e3e13 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:31.608185 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 391/788] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:16:31 2022] POST /identity/v3/endpoints => generated 327 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:32.295525 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1adb5-6ff3-4c04-a9de-2e443b40267a None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:32.296082 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1adb5-6ff3-4c04-a9de-2e443b40267a None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:32.296082 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1adb5-6ff3-4c04-a9de-2e443b40267a None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:32.297575 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 398/789] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:32.302925 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb878a22-f47e-48f4-96ca-1301a259833f None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:32.303257 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb878a22-f47e-48f4-96ca-1301a259833f None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:32.303537 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb878a22-f47e-48f4-96ca-1301a259833f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:32.323255 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-eb878a22-f47e-48f4-96ca-1301a259833f None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:32.324459 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-eb878a22-f47e-48f4-96ca-1301a259833f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:32.338998 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:32.405069 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 392/790] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:32.613806 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08accd75-05c3-4821-b7d0-d986d538a35e None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:32.614075 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08accd75-05c3-4821-b7d0-d986d538a35e None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:32.614390 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08accd75-05c3-4821-b7d0-d986d538a35e None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:32.615944 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 399/791] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:32.621106 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a10cb8-6bd4-4402-97cd-8f08a2f63520 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:32.621585 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a10cb8-6bd4-4402-97cd-8f08a2f63520 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:32.621681 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a10cb8-6bd4-4402-97cd-8f08a2f63520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:32.639970 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-79a10cb8-6bd4-4402-97cd-8f08a2f63520 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:32.641151 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-79a10cb8-6bd4-4402-97cd-8f08a2f63520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:32.678449 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08accd75-05c3-4821-b7d0-d986d538a35e None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:32.712424 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 393/792] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:32.783183 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08accd75-05c3-4821-b7d0-d986d538a35e None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:32.817569 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88ccbf-ba4a-4eff-a698-9e83a081d9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:32.818206 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88ccbf-ba4a-4eff-a698-9e83a081d9c4 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:32.818478 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88ccbf-ba4a-4eff-a698-9e83a081d9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:32.818821 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88ccbf-ba4a-4eff-a698-9e83a081d9c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:32.819956 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88ccbf-ba4a-4eff-a698-9e83a081d9c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:32.821397 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88ccbf-ba4a-4eff-a698-9e83a081d9c4 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:32.823158 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 400/793] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:32.828989 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a10cb8-6bd4-4402-97cd-8f08a2f63520 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:32.910247 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:32.937467 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a10cb8-6bd4-4402-97cd-8f08a2f63520 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:32.972206 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b736038d-f072-466e-9cad-99e95c337437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:32.972820 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736038d-f072-466e-9cad-99e95c337437 None admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:32.973097 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736038d-f072-466e-9cad-99e95c337437 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:32.973328 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736038d-f072-466e-9cad-99e95c337437 None admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:32.974012 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b736038d-f072-466e-9cad-99e95c337437 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:32.975583 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b736038d-f072-466e-9cad-99e95c337437 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:32.987089 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88ccbf-ba4a-4eff-a698-9e83a081d9c4 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:33.001099 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 394/794] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 07:16:32 2022] POST /identity/v3/users => generated 285 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:33.063473 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:33.113548 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88ccbf-ba4a-4eff-a698-9e83a081d9c4 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:33.158764 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c7c69-24f2-4dae-9e8b-353a5bf1467f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:33.159420 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c7c69-24f2-4dae-9e8b-353a5bf1467f None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:33.159738 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c7c69-24f2-4dae-9e8b-353a5bf1467f None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:33.160011 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c7c69-24f2-4dae-9e8b-353a5bf1467f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:33.172930 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0c7c69-24f2-4dae-9e8b-353a5bf1467f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:33.177635 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-be0c7c69-24f2-4dae-9e8b-353a5bf1467f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 07:16:33.177635 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:33.177635 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:33.177635 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:33.177635 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:33.177635 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:33.177635 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:33.177635 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:33.177635 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:33.177635 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:33.177635 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:33.177635 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:33.177635 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:33.177635 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:33.177635 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:33.177635 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:33.179257 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:33.179257 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:33.179257 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:33.179257 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:33.179257 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:33.179257 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:33.179257 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:33.179257 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:33.179257 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:33.179257 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:33.179257 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:33.179257 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:33.179257 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:33.179257 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:33.179257 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:33.179257 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:33.180301 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:33.180301 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:33.180301 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:33.180301 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:33.180301 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:33.180301 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:33.180301 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:33.180301 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:33.180301 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:33.180301 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:33.180301 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 07:16:33.180301 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:33.180301 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 401/795] 10.222.0.30 () {56 vars in 1120 bytes} [Wed Nov 2 07:16:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:33.185796 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b736038d-f072-466e-9cad-99e95c337437 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:33.311192 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b736038d-f072-466e-9cad-99e95c337437 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:33.345823 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:33.354962 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee308f-d1cf-4431-afc6-c51bf4c7efbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:33.356318 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee308f-d1cf-4431-afc6-c51bf4c7efbf None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:33.356929 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee308f-d1cf-4431-afc6-c51bf4c7efbf None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:33.357422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee308f-d1cf-4431-afc6-c51bf4c7efbf None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:33.358474 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee308f-d1cf-4431-afc6-c51bf4c7efbf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:33.358808 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee308f-d1cf-4431-afc6-c51bf4c7efbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:33.360203 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee308f-d1cf-4431-afc6-c51bf4c7efbf None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:33.376648 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 395/796] 10.222.0.30 () {56 vars in 1115 bytes} [Wed Nov 2 07:16:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:33.378392 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c7c69-24f2-4dae-9e8b-353a5bf1467f None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:33.512588 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c7c69-24f2-4dae-9e8b-353a5bf1467f None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:33.534681 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:33.555944 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb888ad2-a581-408a-b34d-8d879a8bf2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:33.556877 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb888ad2-a581-408a-b34d-8d879a8bf2a4 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:33.557326 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb888ad2-a581-408a-b34d-8d879a8bf2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:33.557791 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb888ad2-a581-408a-b34d-8d879a8bf2a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:33.569098 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb888ad2-a581-408a-b34d-8d879a8bf2a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:33.571509 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb888ad2-a581-408a-b34d-8d879a8bf2a4 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:33.593961 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-eb888ad2-a581-408a-b34d-8d879a8bf2a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:33.593961 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:33.593961 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:33.593961 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:33.593961 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:33.593961 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:33.593961 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:33.593961 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:33.593961 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:33.593961 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:33.593961 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:33.593961 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:33.593961 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:16:33.593961 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:16:33.593961 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:16:33.593961 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:16:33.595317 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:33.595317 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:33.595317 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:33.595317 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:33.595317 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:33.595317 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:33.595317 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:33.595317 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:33.595317 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:33.595317 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:33.595317 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:33.595317 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:33.595317 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:33.595317 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:33.595317 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:33.595317 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:33.595317 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:16:33.596213 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:16:33.596213 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:33.596213 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:33.601135 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 402/797] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:33.609337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee308f-d1cf-4431-afc6-c51bf4c7efbf None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:33.753277 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee308f-d1cf-4431-afc6-c51bf4c7efbf None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:33.790753 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8677bb-e36a-488a-a022-0405e5f86a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:33.791274 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8677bb-e36a-488a-a022-0405e5f86a9f None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:33.791550 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8677bb-e36a-488a-a022-0405e5f86a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:33.793319 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8677bb-e36a-488a-a022-0405e5f86a9f None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:33.793319 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8677bb-e36a-488a-a022-0405e5f86a9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:33.793319 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8677bb-e36a-488a-a022-0405e5f86a9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:33.794096 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8677bb-e36a-488a-a022-0405e5f86a9f None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:33.808532 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 396/798] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:16:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:33.811038 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcef9ab-134a-45e4-9b22-25f435572c07 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:33.811317 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcef9ab-134a-45e4-9b22-25f435572c07 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:33.811581 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcef9ab-134a-45e4-9b22-25f435572c07 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:33.813044 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 403/799] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:33.814196 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8677bb-e36a-488a-a022-0405e5f86a9f None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:33.817768 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f23f6-2349-4c72-b67f-8963e34a94ed None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:33.818027 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f23f6-2349-4c72-b67f-8963e34a94ed None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:33.818292 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f23f6-2349-4c72-b67f-8963e34a94ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:33.836105 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-d01f23f6-2349-4c72-b67f-8963e34a94ed None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:33.837277 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-d01f23f6-2349-4c72-b67f-8963e34a94ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:33.874105 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:33.914712 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 404/800] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:16:33.931835 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8677bb-e36a-488a-a022-0405e5f86a9f None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:33.965561 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97987aed-9569-419a-9e81-6cdb57d9e940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:33.966195 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97987aed-9569-419a-9e81-6cdb57d9e940 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:33.966467 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97987aed-9569-419a-9e81-6cdb57d9e940 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:33.966758 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97987aed-9569-419a-9e81-6cdb57d9e940 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:34.094787 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97987aed-9569-419a-9e81-6cdb57d9e940 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:34.096174 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97987aed-9569-419a-9e81-6cdb57d9e940 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:34.108581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-97987aed-9569-419a-9e81-6cdb57d9e940 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:16:34.108581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:34.108581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:34.108581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:34.108581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:34.108581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:34.108581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:34.108581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:34.108581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:34.108581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:34.108581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:34.108581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:34.108581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:16:34.108581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:16:34.108581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:16:34.108581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:16:34.109508 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:34.109508 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:34.109508 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:16:34.109508 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:16:34.109508 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:16:34.109508 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:16:34.109508 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:34.109508 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:34.109508 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:34.109508 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:34.109508 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:34.109508 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:34.109508 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:34.109508 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:34.109508 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:34.109508 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:34.109508 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:34.109508 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:34.110445 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:34.110445 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:34.110445 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:16:34.110445 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:16:34.110445 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:16:34.110445 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:16:34.110445 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:16:34.110445 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:16:34.110445 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:16:34.110445 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:34.110445 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 397/801] 10.222.0.30 () {56 vars in 1093 bytes} [Wed Nov 2 07:16:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:34.115727 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01f23f6-2349-4c72-b67f-8963e34a94ed None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:34.183454 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97987aed-9569-419a-9e81-6cdb57d9e940 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:34.218913 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:34.222043 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01f23f6-2349-4c72-b67f-8963e34a94ed None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:34.257304 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dbc60f-a335-4764-ac81-28a18df0e1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:34.257923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbc60f-a335-4764-ac81-28a18df0e1f0 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:34.258200 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbc60f-a335-4764-ac81-28a18df0e1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:34.258465 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbc60f-a335-4764-ac81-28a18df0e1f0 None admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:34.259218 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dbc60f-a335-4764-ac81-28a18df0e1f0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:34.259521 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dbc60f-a335-4764-ac81-28a18df0e1f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:34.260964 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dbc60f-a335-4764-ac81-28a18df0e1f0 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:34.279018 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 405/802] 10.222.0.30 () {56 vars in 1133 bytes} [Wed Nov 2 07:16:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:34.284802 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dbc60f-a335-4764-ac81-28a18df0e1f0 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:34.295435 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97987aed-9569-419a-9e81-6cdb57d9e940 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:34.309261 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:34.333038 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c792a-4f2a-4be1-9303-191765d60269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:34.333642 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c792a-4f2a-4be1-9303-191765d60269 None admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:34.333912 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c792a-4f2a-4be1-9303-191765d60269 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:34.334182 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c792a-4f2a-4be1-9303-191765d60269 None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:34.335074 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c792a-4f2a-4be1-9303-191765d60269 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:34.336608 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c792a-4f2a-4be1-9303-191765d60269 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:34.346036 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.sql.core [None req-282c792a-4f2a-4be1-9303-191765d60269 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 07:16:34.346036 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 07:16:34.346036 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [parameters: {'id': '5cca14edc74b4d03981eae086102202e', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Nov 02 07:16:34.346036 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81548) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Nov 02 07:16:34.353941 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-282c792a-4f2a-4be1-9303-191765d60269 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Nov 02 07:16:34.353941 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:34.353941 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:16:34.353941 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:16:34.353941 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:16:34.353941 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:16:34.353941 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:16:34.353941 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:16:34.353941 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:16:34.353941 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:16:34.353941 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:16:34.353941 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:16:34.353941 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:16:34.353941 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application result.read() Nov 02 07:16:34.354774 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:16:34.354774 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:16:34.354774 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:16:34.354774 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:16:34.354774 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:16:34.354774 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:16:34.354774 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:16:34.354774 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:16:34.354774 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 07:16:34.354774 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:34.354774 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Nov 02 07:16:34.354774 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:34.354774 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:34.354774 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Nov 02 07:16:34.354774 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return method(*args, **kwargs) Nov 02 07:16:34.354774 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Nov 02 07:16:34.354774 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return project_ref.to_dict() Nov 02 07:16:34.354774 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:16:34.355695 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:16:34.355695 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Nov 02 07:16:34.355695 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application yield resource Nov 02 07:16:34.355695 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 07:16:34.355695 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application next(self.gen) Nov 02 07:16:34.355695 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Nov 02 07:16:34.355695 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self.session.rollback() Nov 02 07:16:34.355695 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Nov 02 07:16:34.355695 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self.force_reraise() Nov 02 07:16:34.355695 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Nov 02 07:16:34.355695 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise self.value Nov 02 07:16:34.355695 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Nov 02 07:16:34.355695 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Nov 02 07:16:34.355695 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Nov 02 07:16:34.355695 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application session.commit() Nov 02 07:16:34.355695 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Nov 02 07:16:34.355695 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Nov 02 07:16:34.356555 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Nov 02 07:16:34.356555 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._prepare_impl() Nov 02 07:16:34.356555 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Nov 02 07:16:34.356555 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self.session.flush() Nov 02 07:16:34.356555 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Nov 02 07:16:34.356555 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._flush(objects) Nov 02 07:16:34.356555 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Nov 02 07:16:34.356555 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Nov 02 07:16:34.356555 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Nov 02 07:16:34.356555 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application compat.raise_( Nov 02 07:16:34.356555 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:16:34.356555 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception Nov 02 07:16:34.356555 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Nov 02 07:16:34.356555 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application flush_context.execute() Nov 02 07:16:34.356555 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Nov 02 07:16:34.356555 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rec.execute(self) Nov 02 07:16:34.356555 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Nov 02 07:16:34.357456 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Nov 02 07:16:34.357456 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Nov 02 07:16:34.357456 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application _emit_insert_statements( Nov 02 07:16:34.357456 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Nov 02 07:16:34.357456 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application c = connection._execute_20( Nov 02 07:16:34.357456 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Nov 02 07:16:34.357456 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Nov 02 07:16:34.357456 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Nov 02 07:16:34.357456 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Nov 02 07:16:34.357456 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Nov 02 07:16:34.357456 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ret = self._execute_context( Nov 02 07:16:34.357456 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Nov 02 07:16:34.357456 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Nov 02 07:16:34.357456 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Nov 02 07:16:34.357456 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Nov 02 07:16:34.358243 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 07:16:34.358243 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception Nov 02 07:16:34.358243 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 07:16:34.358243 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 07:16:34.358243 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 07:16:34.358243 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 07:16:34.358243 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 07:16:34.358243 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 07:16:34.358243 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 07:16:34.358243 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application conn.query(q) Nov 02 07:16:34.358243 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 07:16:34.358243 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 07:16:34.358243 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 07:16:34.358243 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application result.read() Nov 02 07:16:34.358243 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 07:16:34.358243 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 07:16:34.358243 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 07:16:34.359137 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 07:16:34.359137 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 07:16:34.359137 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 07:16:34.359137 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 07:16:34.359137 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 07:16:34.359137 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 07:16:34.359137 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 07:16:34.359137 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [parameters: {'id': '5cca14edc74b4d03981eae086102202e', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Nov 02 07:16:34.359137 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Nov 02 07:16:34.359137 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:34.359137 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:34.359137 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:34.359137 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:34.359137 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Nov 02 07:16:34.359137 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Nov 02 07:16:34.359901 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Nov 02 07:16:34.359901 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Nov 02 07:16:34.359901 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Nov 02 07:16:34.359901 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:34.359901 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:16:34.359901 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:34.359901 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:34.359901 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:34.359901 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:34.359901 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:34.359901 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:34.359901 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:34.359901 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:34.359901 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:34.359901 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:34.359901 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:34.359901 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:34.360802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Nov 02 07:16:34.360802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Nov 02 07:16:34.360802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:34.360802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:34.360802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Nov 02 07:16:34.360802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Nov 02 07:16:34.360802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:34.360802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:34.360802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Nov 02 07:16:34.360802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.Conflict( Nov 02 07:16:34.360802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Nov 02 07:16:34.360802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:34.360802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 398/803] 10.222.0.30 () {60 vars in 1137 bytes} [Wed Nov 2 07:16:34 2022] POST /identity/v3/domains => generated 190 bytes in 172 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:34.361483 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c792a-4f2a-4be1-9303-191765d60269 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:34.395795 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dbc60f-a335-4764-ac81-28a18df0e1f0 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:34.431324 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a11156-ecc8-47bc-9202-e1ad7c932234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:34.432037 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a11156-ecc8-47bc-9202-e1ad7c932234 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:34.432313 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a11156-ecc8-47bc-9202-e1ad7c932234 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:34.432577 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a11156-ecc8-47bc-9202-e1ad7c932234 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:34.444063 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a11156-ecc8-47bc-9202-e1ad7c932234 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:34.445505 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a11156-ecc8-47bc-9202-e1ad7c932234 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:34.459852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-a8a11156-ecc8-47bc-9202-e1ad7c932234 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:34.459852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:34.459852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:34.459852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:34.459852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:34.459852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:34.459852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:34.459852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:34.459852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:34.459852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:34.459852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:34.459852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:34.459852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:16:34.459852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:16:34.459852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:16:34.459852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:16:34.461059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:34.461059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:34.461059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:34.461059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:34.461059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:34.461059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:34.461059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:34.461059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:34.461059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:34.461059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:34.461059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:34.461059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:34.461059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:34.461059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:34.461059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:34.461059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:34.461059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:16:34.462122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:16:34.462122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:34.462122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:34.462122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 406/804] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:34.466550 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a11156-ecc8-47bc-9202-e1ad7c932234 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:34.468413 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c792a-4f2a-4be1-9303-191765d60269 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:34.480278 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:34.505438 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d2b70-8b56-4ed0-a42e-94595e10bcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:34.507675 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d2b70-8b56-4ed0-a42e-94595e10bcd8 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:34.507947 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d2b70-8b56-4ed0-a42e-94595e10bcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:34.508205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d2b70-8b56-4ed0-a42e-94595e10bcd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:34.519282 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4d2b70-8b56-4ed0-a42e-94595e10bcd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:34.521093 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4d2b70-8b56-4ed0-a42e-94595e10bcd8 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:34.537458 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-0b4d2b70-8b56-4ed0-a42e-94595e10bcd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:34.537458 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:34.537458 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:34.537458 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:34.537458 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:34.537458 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:34.537458 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:34.537458 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:34.537458 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:34.537458 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:34.537458 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:34.537458 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:34.537458 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:16:34.537458 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:16:34.537458 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:16:34.537458 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:16:34.538423 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:34.538423 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:34.538423 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:34.538423 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:34.538423 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:34.538423 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:34.538423 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:34.538423 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:34.538423 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:34.538423 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:34.538423 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:34.538423 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:34.538423 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:34.538423 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:34.538423 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:34.538423 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:34.538423 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:16:34.539349 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:16:34.539349 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:34.539349 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:34.539349 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 399/805] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:34.544583 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d2b70-8b56-4ed0-a42e-94595e10bcd8 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:34.575412 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:34.582361 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a11156-ecc8-47bc-9202-e1ad7c932234 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:34.623040 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd4f742-32d8-4388-829b-346a4fa210b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:34.624431 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4f742-32d8-4388-829b-346a4fa210b4 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:34.624713 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4f742-32d8-4388-829b-346a4fa210b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:34.624987 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4f742-32d8-4388-829b-346a4fa210b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:34.625633 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd4f742-32d8-4388-829b-346a4fa210b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:34.625935 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd4f742-32d8-4388-829b-346a4fa210b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:34.627345 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd4f742-32d8-4388-829b-346a4fa210b4 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:34.647345 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 407/806] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:16:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:34.648583 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd4f742-32d8-4388-829b-346a4fa210b4 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:34.652692 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d2b70-8b56-4ed0-a42e-94595e10bcd8 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:34.689521 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f997182-b58b-40df-8b31-8ff89ebee5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:34.690130 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f997182-b58b-40df-8b31-8ff89ebee5f7 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:34.690405 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f997182-b58b-40df-8b31-8ff89ebee5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:34.690697 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f997182-b58b-40df-8b31-8ff89ebee5f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:34.691338 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f997182-b58b-40df-8b31-8ff89ebee5f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:34.691638 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f997182-b58b-40df-8b31-8ff89ebee5f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:34.693453 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f997182-b58b-40df-8b31-8ff89ebee5f7 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:34.718681 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 400/807] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:16:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:34.722422 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:34.758817 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd4f742-32d8-4388-829b-346a4fa210b4 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:34.797670 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a155b-ef62-498d-92b7-60ca43b08c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:34.798320 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a155b-ef62-498d-92b7-60ca43b08c43 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:34.798593 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a155b-ef62-498d-92b7-60ca43b08c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:34.798892 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a155b-ef62-498d-92b7-60ca43b08c43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:34.806688 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3a155b-ef62-498d-92b7-60ca43b08c43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:34.808171 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3a155b-ef62-498d-92b7-60ca43b08c43 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:34.821042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-2e3a155b-ef62-498d-92b7-60ca43b08c43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:16:34.821042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:34.821042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:34.821042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:34.821042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:34.821042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:34.821042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:34.821042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:34.821042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:34.821042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:34.821042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:34.821042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:34.821042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:16:34.821042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:16:34.821042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:16:34.822000 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:16:34.822000 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:34.822000 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:34.822000 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:34.822000 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:34.822000 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:34.822000 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:34.822000 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:34.822000 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:34.822000 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:34.822000 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:34.822000 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:34.822000 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:34.822000 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:34.822000 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:34.822000 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:34.822000 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:34.822914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:16:34.822914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:16:34.822914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:16:34.822914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:16:34.822914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:16:34.822914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:16:34.822914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:16:34.822914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:34.822914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 408/808] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:16:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:34.828256 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f997182-b58b-40df-8b31-8ff89ebee5f7 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:34.927258 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:34.940841 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f997182-b58b-40df-8b31-8ff89ebee5f7 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:34.977963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff96a074-cbd0-41be-a8f7-b567ba14a838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:34.978568 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96a074-cbd0-41be-a8f7-b567ba14a838 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:34.978917 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96a074-cbd0-41be-a8f7-b567ba14a838 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:34.979172 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96a074-cbd0-41be-a8f7-b567ba14a838 None admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:34.979861 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff96a074-cbd0-41be-a8f7-b567ba14a838 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:34.980154 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff96a074-cbd0-41be-a8f7-b567ba14a838 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:34.981571 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff96a074-cbd0-41be-a8f7-b567ba14a838 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:34.993863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 401/809] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 07:16:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:35.000059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a155b-ef62-498d-92b7-60ca43b08c43 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:35.106004 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a155b-ef62-498d-92b7-60ca43b08c43 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:35.135189 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:35.142368 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302efdf-8be7-4f1b-adcc-003bc8a85d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:35.143090 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302efdf-8be7-4f1b-adcc-003bc8a85d07 None admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:35.143365 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302efdf-8be7-4f1b-adcc-003bc8a85d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:35.143639 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302efdf-8be7-4f1b-adcc-003bc8a85d07 None admin] PATH_INFO: `/v3/projects/25e2bc8417754f909b416b93d5b7afb7/users/e6495385dccc4fa48c05b1feb957846c/roles/e6483a4fca514243bed809857f555411` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:35.145923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4302efdf-8be7-4f1b-adcc-003bc8a85d07 None admin] RBAC: Authorizing `identity:create_grant(project_id=25e2bc8417754f909b416b93d5b7afb7, user_id=e6495385dccc4fa48c05b1feb957846c, role_id=e6483a4fca514243bed809857f555411)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:35.147706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4302efdf-8be7-4f1b-adcc-003bc8a85d07 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:35.165525 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 409/810] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 07:16:34 2022] PUT /identity/v3/projects/25e2bc8417754f909b416b93d5b7afb7/users/e6495385dccc4fa48c05b1feb957846c/roles/e6483a4fca514243bed809857f555411 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:16:35.263928 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdadcff-83fb-405c-aae5-a637ae050369 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:35.264116 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdadcff-83fb-405c-aae5-a637ae050369 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:35.264366 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdadcff-83fb-405c-aae5-a637ae050369 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:35.265729 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 402/811] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:35.271673 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ca029-79e8-4d5b-abcf-1b18d1d8f571 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:35.271909 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ca029-79e8-4d5b-abcf-1b18d1d8f571 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:35.272216 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ca029-79e8-4d5b-abcf-1b18d1d8f571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:35.292116 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-d77ca029-79e8-4d5b-abcf-1b18d1d8f571 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:35.293351 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-d77ca029-79e8-4d5b-abcf-1b18d1d8f571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:35.364931 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 410/812] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:35.625167 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdadcff-83fb-405c-aae5-a637ae050369 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:35.724253 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdadcff-83fb-405c-aae5-a637ae050369 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:35.758857 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b28a22-3fdc-49a8-92ff-a18efa7ab163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:35.759470 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b28a22-3fdc-49a8-92ff-a18efa7ab163 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:35.759741 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b28a22-3fdc-49a8-92ff-a18efa7ab163 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:35.760001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b28a22-3fdc-49a8-92ff-a18efa7ab163 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:35.770233 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b28a22-3fdc-49a8-92ff-a18efa7ab163 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:35.775492 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-33b28a22-3fdc-49a8-92ff-a18efa7ab163 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:35.775492 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:35.775492 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:35.775492 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:35.775492 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:35.775492 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:35.775492 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:35.775492 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:35.775492 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:35.775492 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:35.775492 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:35.775492 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:35.775492 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:35.775492 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:35.775492 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:35.775492 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:35.776436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:35.776436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:35.776436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:35.776436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:35.776436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:35.776436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:35.776436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:35.776436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:35.776436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:35.776436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:35.776436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:35.776436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:35.776436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:35.776436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:35.776436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:35.776436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:35.777303 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:35.777303 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:35.777303 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:35.777303 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:35.777303 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:35.777303 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:35.777303 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:35.777303 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:35.777303 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:35.777303 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:35.777303 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:35.777303 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:35.777303 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 403/813] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 07:16:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:35.782983 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ca029-79e8-4d5b-abcf-1b18d1d8f571 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:35.889735 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ca029-79e8-4d5b-abcf-1b18d1d8f571 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:35.910498 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:35.926444 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc49b73a-a0b3-41d3-8e9a-c8e8c61a9851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:35.927092 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49b73a-a0b3-41d3-8e9a-c8e8c61a9851 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:35.927363 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49b73a-a0b3-41d3-8e9a-c8e8c61a9851 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:35.927667 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49b73a-a0b3-41d3-8e9a-c8e8c61a9851 None admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:35.928319 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc49b73a-a0b3-41d3-8e9a-c8e8c61a9851 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:35.928615 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc49b73a-a0b3-41d3-8e9a-c8e8c61a9851 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:35.929986 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc49b73a-a0b3-41d3-8e9a-c8e8c61a9851 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:35.938062 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 411/814] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:35.944197 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b28a22-3fdc-49a8-92ff-a18efa7ab163 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:35.955826 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:36.050888 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b28a22-3fdc-49a8-92ff-a18efa7ab163 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:36.086139 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1804b-322a-46b5-9d65-edd853538f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:36.086783 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1804b-322a-46b5-9d65-edd853538f10 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:36.087056 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1804b-322a-46b5-9d65-edd853538f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:36.087320 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1804b-322a-46b5-9d65-edd853538f10 None admin] PATH_INFO: `/v3/users/07b42d947cd34b3ca6508ce6c2b143a7` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:36.100129 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac1804b-322a-46b5-9d65-edd853538f10 None admin] RBAC: Authorizing `identity:get_user(user_id=07b42d947cd34b3ca6508ce6c2b143a7)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:36.101557 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac1804b-322a-46b5-9d65-edd853538f10 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:36.103419 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 404/815] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:35 2022] GET /identity/v3/users/07b42d947cd34b3ca6508ce6c2b143a7 => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:36.109126 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc49b73a-a0b3-41d3-8e9a-c8e8c61a9851 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:36.215310 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc49b73a-a0b3-41d3-8e9a-c8e8c61a9851 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:36.250338 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3ddbe-8948-4a7e-ae29-b679beff412e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:36.251002 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3ddbe-8948-4a7e-ae29-b679beff412e None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:36.251276 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3ddbe-8948-4a7e-ae29-b679beff412e None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:36.251547 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3ddbe-8948-4a7e-ae29-b679beff412e None admin] PATH_INFO: `/v3/projects/3b537ba94af04c8496b7251223333790` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:36.252705 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe3ddbe-8948-4a7e-ae29-b679beff412e None admin] RBAC: Authorizing `identity:get_project(project_id=3b537ba94af04c8496b7251223333790)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:36.254149 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe3ddbe-8948-4a7e-ae29-b679beff412e None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:36.256046 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 412/816] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:36 2022] GET /identity/v3/projects/3b537ba94af04c8496b7251223333790 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:36.261906 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1804b-322a-46b5-9d65-edd853538f10 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:36.329066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:36.363024 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1804b-322a-46b5-9d65-edd853538f10 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:36.396205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122910a1-4d80-43bd-a034-cb9418010c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:36.396880 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122910a1-4d80-43bd-a034-cb9418010c28 None admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:36.397155 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122910a1-4d80-43bd-a034-cb9418010c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:36.397423 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122910a1-4d80-43bd-a034-cb9418010c28 None admin] PATH_INFO: `/v3/projects/3b537ba94af04c8496b7251223333790/users/07b42d947cd34b3ca6508ce6c2b143a7/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:36.399690 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122910a1-4d80-43bd-a034-cb9418010c28 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b537ba94af04c8496b7251223333790, user_id=07b42d947cd34b3ca6508ce6c2b143a7, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:36.401120 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122910a1-4d80-43bd-a034-cb9418010c28 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:36.415083 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 405/817] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 07:16:36 2022] PUT /identity/v3/projects/3b537ba94af04c8496b7251223333790/users/07b42d947cd34b3ca6508ce6c2b143a7/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:16:37.293198 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e3990-8a2a-411e-ab75-42d9f020f56f None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:37.293832 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e3990-8a2a-411e-ab75-42d9f020f56f None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:37.293832 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e3990-8a2a-411e-ab75-42d9f020f56f None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:37.295100 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 413/818] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:37.300431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f15ce-5c03-4081-855c-ee433f3572bd None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:37.300897 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f15ce-5c03-4081-855c-ee433f3572bd None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:37.300988 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f15ce-5c03-4081-855c-ee433f3572bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:37.319367 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-976f15ce-5c03-4081-855c-ee433f3572bd None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:37.320547 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-976f15ce-5c03-4081-855c-ee433f3572bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:37.388651 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 406/819] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:37.654983 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e3990-8a2a-411e-ab75-42d9f020f56f None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:37.711686 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:37.763375 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e3990-8a2a-411e-ab75-42d9f020f56f None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:37.798694 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49587eb3-e91b-479c-8ddb-e56fd42b2885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:37.799414 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49587eb3-e91b-479c-8ddb-e56fd42b2885 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:37.799684 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49587eb3-e91b-479c-8ddb-e56fd42b2885 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:37.799914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49587eb3-e91b-479c-8ddb-e56fd42b2885 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:37.810241 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49587eb3-e91b-479c-8ddb-e56fd42b2885 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:37.815012 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-49587eb3-e91b-479c-8ddb-e56fd42b2885 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 07:16:37.815012 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:37.815012 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:37.815012 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:37.815012 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:37.815012 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:37.815012 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:37.815012 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:37.815012 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:37.815012 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:37.815012 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:37.815012 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:37.815012 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:37.815012 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:37.815012 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:37.815012 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:37.816231 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:37.816231 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:37.816231 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:37.816231 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:37.816231 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:37.816231 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:37.816231 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:37.816231 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:37.816231 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:37.816231 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:37.816231 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:37.816231 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:37.816231 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:37.816231 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:37.816231 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:37.816231 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:37.817152 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:37.817152 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:37.817152 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:37.817152 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:37.817152 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:37.817152 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:37.817152 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:37.817152 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:37.817152 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:37.817152 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:37.817152 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 07:16:37.817152 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:37.817152 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 414/820] 10.222.0.30 () {56 vars in 1120 bytes} [Wed Nov 2 07:16:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:37.822363 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f15ce-5c03-4081-855c-ee433f3572bd None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:37.881027 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:37.929031 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f15ce-5c03-4081-855c-ee433f3572bd None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:37.962627 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa341e-e114-4b71-a6e8-ec5b37de8f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:37.963254 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa341e-e114-4b71-a6e8-ec5b37de8f1f None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:37.963530 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa341e-e114-4b71-a6e8-ec5b37de8f1f None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:37.963775 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa341e-e114-4b71-a6e8-ec5b37de8f1f None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:37.964430 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aa341e-e114-4b71-a6e8-ec5b37de8f1f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:37.964725 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aa341e-e114-4b71-a6e8-ec5b37de8f1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:37.966087 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aa341e-e114-4b71-a6e8-ec5b37de8f1f None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:37.973887 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 407/821] 10.222.0.30 () {56 vars in 1115 bytes} [Wed Nov 2 07:16:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:37.980109 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49587eb3-e91b-479c-8ddb-e56fd42b2885 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:38.084516 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49587eb3-e91b-479c-8ddb-e56fd42b2885 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:38.095394 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:38.120645 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7da2e-4616-40c0-b617-4390112d80bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:38.121275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7da2e-4616-40c0-b617-4390112d80bb None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:38.121550 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7da2e-4616-40c0-b617-4390112d80bb None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:38.121823 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7da2e-4616-40c0-b617-4390112d80bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:38.132352 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad7da2e-4616-40c0-b617-4390112d80bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:38.133738 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad7da2e-4616-40c0-b617-4390112d80bb None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:38.148715 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-6ad7da2e-4616-40c0-b617-4390112d80bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:38.148715 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:38.148715 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:38.148715 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:38.148715 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:38.148715 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:38.148715 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:38.148715 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:38.148715 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:38.148715 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:38.148715 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:38.148715 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:38.148715 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:16:38.148715 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:16:38.148715 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:16:38.148715 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:16:38.149836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:38.149836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:38.149836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:38.149836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:38.149836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:38.149836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:38.149836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:38.149836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:38.149836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:38.149836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:38.149836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:38.149836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:38.149836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:38.149836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:38.149836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:38.149836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:38.149836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:16:38.150779 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:16:38.150779 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:38.150779 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:38.150779 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 415/822] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:38.155840 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa341e-e114-4b71-a6e8-ec5b37de8f1f None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:38.258215 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa341e-e114-4b71-a6e8-ec5b37de8f1f None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:38.275925 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:38.295235 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5cd283-6d23-4d3c-a0f7-f0d1ed62024a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:38.295854 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5cd283-6d23-4d3c-a0f7-f0d1ed62024a None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:38.296132 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5cd283-6d23-4d3c-a0f7-f0d1ed62024a None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:38.296400 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5cd283-6d23-4d3c-a0f7-f0d1ed62024a None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:38.297027 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5cd283-6d23-4d3c-a0f7-f0d1ed62024a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:38.297341 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5cd283-6d23-4d3c-a0f7-f0d1ed62024a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:38.298741 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5cd283-6d23-4d3c-a0f7-f0d1ed62024a None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:38.310870 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 408/823] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:16:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:38.316746 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7da2e-4616-40c0-b617-4390112d80bb None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:38.418344 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7da2e-4616-40c0-b617-4390112d80bb None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:38.456831 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcafe0c-5273-482a-adf4-27d5d90d03f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:38.457498 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcafe0c-5273-482a-adf4-27d5d90d03f5 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:38.457778 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcafe0c-5273-482a-adf4-27d5d90d03f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:38.458048 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcafe0c-5273-482a-adf4-27d5d90d03f5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:38.466194 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcafe0c-5273-482a-adf4-27d5d90d03f5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:38.467678 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcafe0c-5273-482a-adf4-27d5d90d03f5 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:38.479122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-ebcafe0c-5273-482a-adf4-27d5d90d03f5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:16:38.479122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:38.479122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:38.479122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:38.479122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:38.479122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:38.479122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:38.479122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:38.479122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:38.479122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:38.479122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:38.479122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:38.479122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:16:38.479122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:16:38.479122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:16:38.479122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:16:38.480125 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:38.480125 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:38.480125 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:16:38.480125 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:16:38.480125 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:16:38.480125 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:16:38.480125 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:38.480125 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:38.480125 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:38.480125 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:38.480125 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:38.480125 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:38.480125 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:38.480125 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:38.480125 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:38.480125 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:38.480125 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:38.480125 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:38.481050 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:38.481050 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:38.481050 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:16:38.481050 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:16:38.481050 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:16:38.481050 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:16:38.481050 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:16:38.481050 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:16:38.481050 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 07:16:38.481050 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:38.481050 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 416/824] 10.222.0.30 () {56 vars in 1093 bytes} [Wed Nov 2 07:16:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:38.486150 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5cd283-6d23-4d3c-a0f7-f0d1ed62024a None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:38.535980 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36394ce2-9535-42ee-9da6-b3909a4051b2 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:38.536214 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36394ce2-9535-42ee-9da6-b3909a4051b2 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:38.536507 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36394ce2-9535-42ee-9da6-b3909a4051b2 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:38.537938 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 417/825] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:16:38.542985 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fef892-38d7-47b4-a08a-8d6697fd55bc None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:38.543223 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fef892-38d7-47b4-a08a-8d6697fd55bc None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:38.543517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fef892-38d7-47b4-a08a-8d6697fd55bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:38.562899 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-67fef892-38d7-47b4-a08a-8d6697fd55bc None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:38.564122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-67fef892-38d7-47b4-a08a-8d6697fd55bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:38.581394 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:38.587936 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5cd283-6d23-4d3c-a0f7-f0d1ed62024a None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:38.622380 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b138ef7-6010-4ddd-ac3b-b821999ef28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:38.623012 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b138ef7-6010-4ddd-ac3b-b821999ef28c None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:38.623290 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b138ef7-6010-4ddd-ac3b-b821999ef28c None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:38.623551 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b138ef7-6010-4ddd-ac3b-b821999ef28c None admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:38.624283 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b138ef7-6010-4ddd-ac3b-b821999ef28c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:38.624577 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b138ef7-6010-4ddd-ac3b-b821999ef28c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:38.626066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b138ef7-6010-4ddd-ac3b-b821999ef28c None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:38.635562 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 418/826] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:38.644595 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 409/827] 10.222.0.30 () {56 vars in 1133 bytes} [Wed Nov 2 07:16:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:38.650602 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fef892-38d7-47b4-a08a-8d6697fd55bc None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:38.750894 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fef892-38d7-47b4-a08a-8d6697fd55bc None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:38.785708 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b4b44-e69f-4209-b751-6f159dbb299e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:38.786308 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b4b44-e69f-4209-b751-6f159dbb299e None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:38.786560 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b4b44-e69f-4209-b751-6f159dbb299e None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:38.786919 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b4b44-e69f-4209-b751-6f159dbb299e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:38.807789 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:38.855286 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b4b44-e69f-4209-b751-6f159dbb299e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:38.856738 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b4b44-e69f-4209-b751-6f159dbb299e None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:38.899561 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b138ef7-6010-4ddd-ac3b-b821999ef28c None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:38.934320 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:38.956481 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 419/828] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 07:16:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:16:38.962521 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b4b44-e69f-4209-b751-6f159dbb299e None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:38.998942 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b138ef7-6010-4ddd-ac3b-b821999ef28c None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:39.012213 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:39.034114 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e9fb11-f43c-41e7-ae03-48f98acb6b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:39.034761 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9fb11-f43c-41e7-ae03-48f98acb6b63 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:39.035031 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9fb11-f43c-41e7-ae03-48f98acb6b63 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:39.035292 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9fb11-f43c-41e7-ae03-48f98acb6b63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:39.045092 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e9fb11-f43c-41e7-ae03-48f98acb6b63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:39.049968 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-c2e9fb11-f43c-41e7-ae03-48f98acb6b63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:39.049968 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:39.049968 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:39.049968 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:39.049968 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:39.049968 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:39.049968 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:39.049968 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:39.049968 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:39.049968 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:39.049968 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:39.049968 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:39.049968 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:39.049968 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:39.049968 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:39.049968 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:39.051043 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:39.051043 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:39.051043 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:39.051043 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:39.051043 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:39.051043 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:39.051043 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:39.051043 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:39.051043 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:39.051043 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:39.051043 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:39.051043 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:39.051043 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:39.051043 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:39.051043 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:39.051043 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:39.052090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:39.052090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:39.052090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:39.052090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:39.052090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:39.052090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:39.052090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:39.052090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:39.052090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:39.052090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:39.052090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:39.052090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:16:39.052090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 410/829] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 07:16:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:39.057166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e9fb11-f43c-41e7-ae03-48f98acb6b63 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:39.065629 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b4b44-e69f-4209-b751-6f159dbb299e None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:39.103261 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1864b23-f656-4cc3-91d5-baf8e1cc6fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:39.103895 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1864b23-f656-4cc3-91d5-baf8e1cc6fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:39.104169 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1864b23-f656-4cc3-91d5-baf8e1cc6fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:39.104438 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1864b23-f656-4cc3-91d5-baf8e1cc6fd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:39.114188 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1864b23-f656-4cc3-91d5-baf8e1cc6fd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:39.115912 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1864b23-f656-4cc3-91d5-baf8e1cc6fd6 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:39.129990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-e1864b23-f656-4cc3-91d5-baf8e1cc6fd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:39.129990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:39.129990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:39.129990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:39.129990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:39.129990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:39.129990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:39.129990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:39.129990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:39.129990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:39.129990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:39.129990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:39.129990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:16:39.129990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:16:39.129990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:16:39.129990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:16:39.131068 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:39.131068 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:39.131068 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:39.131068 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:39.131068 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:39.131068 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:39.131068 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:39.131068 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:39.131068 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:39.131068 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:39.131068 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:39.131068 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:39.131068 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:39.131068 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:39.131068 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:39.131068 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:39.131068 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:16:39.136897 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:16:39.136897 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:16:39.136897 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:39.136897 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 420/830] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:39.137128 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1864b23-f656-4cc3-91d5-baf8e1cc6fd6 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:39.156489 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:39.163069 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e9fb11-f43c-41e7-ae03-48f98acb6b63 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:39.197940 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595a0fb-db15-4ac0-affb-5e80a5b1b0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:39.198611 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595a0fb-db15-4ac0-affb-5e80a5b1b0ac None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:39.198906 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595a0fb-db15-4ac0-affb-5e80a5b1b0ac None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:39.199172 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595a0fb-db15-4ac0-affb-5e80a5b1b0ac None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:39.199822 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0595a0fb-db15-4ac0-affb-5e80a5b1b0ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:39.200116 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0595a0fb-db15-4ac0-affb-5e80a5b1b0ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:39.201489 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0595a0fb-db15-4ac0-affb-5e80a5b1b0ac None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:39.209749 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 411/831] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:39.215746 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595a0fb-db15-4ac0-affb-5e80a5b1b0ac None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:39.238390 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1864b23-f656-4cc3-91d5-baf8e1cc6fd6 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:39.241628 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:39.276536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a135d3f-0ff9-4e5c-9be1-446a771bff28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:39.277062 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a135d3f-0ff9-4e5c-9be1-446a771bff28 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:39.277341 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a135d3f-0ff9-4e5c-9be1-446a771bff28 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:39.277602 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a135d3f-0ff9-4e5c-9be1-446a771bff28 None admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:39.278226 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a135d3f-0ff9-4e5c-9be1-446a771bff28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:39.278517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a135d3f-0ff9-4e5c-9be1-446a771bff28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:39.279902 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a135d3f-0ff9-4e5c-9be1-446a771bff28 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:39.290850 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 421/832] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:16:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:39.296554 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a135d3f-0ff9-4e5c-9be1-446a771bff28 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:39.320152 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595a0fb-db15-4ac0-affb-5e80a5b1b0ac None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:39.354473 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63728666-8417-46be-897e-5373640d8de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:39.355127 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63728666-8417-46be-897e-5373640d8de4 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:39.355400 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63728666-8417-46be-897e-5373640d8de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:39.355665 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63728666-8417-46be-897e-5373640d8de4 None admin] PATH_INFO: `/v3/users/07b42d947cd34b3ca6508ce6c2b143a7` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:39.356769 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63728666-8417-46be-897e-5373640d8de4 None admin] RBAC: Authorizing `identity:get_user(user_id=07b42d947cd34b3ca6508ce6c2b143a7)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:39.358195 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63728666-8417-46be-897e-5373640d8de4 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:39.359823 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 412/833] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:39 2022] GET /identity/v3/users/07b42d947cd34b3ca6508ce6c2b143a7 => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:39.365938 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63728666-8417-46be-897e-5373640d8de4 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:39.392878 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:39.394275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a135d3f-0ff9-4e5c-9be1-446a771bff28 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:39.427823 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bdb6ac-751b-4e40-9d4d-9de86abfe669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:39.428445 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdb6ac-751b-4e40-9d4d-9de86abfe669 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:39.428714 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdb6ac-751b-4e40-9d4d-9de86abfe669 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:39.428975 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdb6ac-751b-4e40-9d4d-9de86abfe669 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:39.435501 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bdb6ac-751b-4e40-9d4d-9de86abfe669 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:39.437063 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bdb6ac-751b-4e40-9d4d-9de86abfe669 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:39.448164 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-35bdb6ac-751b-4e40-9d4d-9de86abfe669 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:16:39.448164 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:39.448164 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:39.448164 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:39.448164 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:39.448164 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:39.448164 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:39.448164 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:39.448164 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:39.448164 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:39.448164 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:39.448164 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:39.448164 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:16:39.448164 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:16:39.448164 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:16:39.449245 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:16:39.449245 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:39.449245 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:39.449245 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:39.449245 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:39.449245 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:39.449245 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:39.449245 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:39.449245 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:39.449245 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:39.449245 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:39.449245 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:39.449245 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:39.449245 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:39.449245 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:39.449245 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:39.449245 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:39.450169 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:16:39.450169 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:16:39.450169 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:16:39.450169 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:16:39.450169 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:16:39.450169 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:16:39.450169 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:16:39.450169 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:39.450169 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 422/834] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:16:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:16:39.455080 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bdb6ac-751b-4e40-9d4d-9de86abfe669 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:39.465980 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63728666-8417-46be-897e-5373640d8de4 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:39.480316 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:39.501714 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc128c7-8eee-4cbb-ac83-5edb1e4e064e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:39.502422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc128c7-8eee-4cbb-ac83-5edb1e4e064e None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:39.502719 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc128c7-8eee-4cbb-ac83-5edb1e4e064e None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:39.502984 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc128c7-8eee-4cbb-ac83-5edb1e4e064e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:39.552849 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bdb6ac-751b-4e40-9d4d-9de86abfe669 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:39.570150 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc128c7-8eee-4cbb-ac83-5edb1e4e064e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:39.571633 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc128c7-8eee-4cbb-ac83-5edb1e4e064e None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:39.586520 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983e010f-294d-4ccc-aa37-771e47a2d484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:39.587157 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e010f-294d-4ccc-aa37-771e47a2d484 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:39.587429 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e010f-294d-4ccc-aa37-771e47a2d484 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:39.587694 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e010f-294d-4ccc-aa37-771e47a2d484 None admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:39.588376 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983e010f-294d-4ccc-aa37-771e47a2d484 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:39.588674 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983e010f-294d-4ccc-aa37-771e47a2d484 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:39.590093 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983e010f-294d-4ccc-aa37-771e47a2d484 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:39.599841 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:39.603762 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 423/835] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 07:16:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:39.609332 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983e010f-294d-4ccc-aa37-771e47a2d484 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:39.678954 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 413/836] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 07:16:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:16:39.680252 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc128c7-8eee-4cbb-ac83-5edb1e4e064e None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:39.692256 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:39.708919 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983e010f-294d-4ccc-aa37-771e47a2d484 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:39.743948 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd919ee-b23e-4f9c-add8-12164024358b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:39.744543 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd919ee-b23e-4f9c-add8-12164024358b None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:39.745121 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd919ee-b23e-4f9c-add8-12164024358b None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:39.745121 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd919ee-b23e-4f9c-add8-12164024358b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:39.745772 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd919ee-b23e-4f9c-add8-12164024358b None admin] RBAC: Adding query filter params (role.id=e6483a4fca514243bed809857f555411, scope.project.id=25e2bc8417754f909b416b93d5b7afb7, user.id=e6495385dccc4fa48c05b1feb957846c) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:39.746082 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd919ee-b23e-4f9c-add8-12164024358b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:39.747521 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd919ee-b23e-4f9c-add8-12164024358b None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:39.753418 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 424/837] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 07:16:39 2022] GET /identity/v3/role_assignments?user.id=e6495385dccc4fa48c05b1feb957846c&scope.project.id=25e2bc8417754f909b416b93d5b7afb7&role.id=e6483a4fca514243bed809857f555411 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:39.780401 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc128c7-8eee-4cbb-ac83-5edb1e4e064e None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:39.809314 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:39.820256 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb201a-f079-4730-a492-598e499b8486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:39.820909 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb201a-f079-4730-a492-598e499b8486 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:39.821212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb201a-f079-4730-a492-598e499b8486 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:39.821478 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb201a-f079-4730-a492-598e499b8486 None admin] PATH_INFO: `/v3/projects/3b537ba94af04c8496b7251223333790` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:39.822638 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb201a-f079-4730-a492-598e499b8486 None admin] RBAC: Authorizing `identity:get_project(project_id=3b537ba94af04c8496b7251223333790)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:39.824116 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb201a-f079-4730-a492-598e499b8486 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:39.825774 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 414/838] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:39 2022] GET /identity/v3/projects/3b537ba94af04c8496b7251223333790 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:39.832383 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd919ee-b23e-4f9c-add8-12164024358b None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:39.951119 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd919ee-b23e-4f9c-add8-12164024358b None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:39.985127 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f293b8-f498-4c19-8663-dc2213e25c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:39.985740 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f293b8-f498-4c19-8663-dc2213e25c02 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:39.986019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f293b8-f498-4c19-8663-dc2213e25c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:39.986285 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f293b8-f498-4c19-8663-dc2213e25c02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:39.987044 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f293b8-f498-4c19-8663-dc2213e25c02 None admin] RBAC: Adding query filter params (role.id=1e35bb725eb8475ea63e2d9538ab67d5, scope.project.id=3b537ba94af04c8496b7251223333790, user.id=07b42d947cd34b3ca6508ce6c2b143a7) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:39.987348 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f293b8-f498-4c19-8663-dc2213e25c02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:39.988785 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f293b8-f498-4c19-8663-dc2213e25c02 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:39.994928 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 425/839] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 07:16:39 2022] GET /identity/v3/role_assignments?user.id=07b42d947cd34b3ca6508ce6c2b143a7&scope.project.id=3b537ba94af04c8496b7251223333790&role.id=1e35bb725eb8475ea63e2d9538ab67d5 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:42.159796 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc6c7a-10e8-4520-82e2-e8c504c781b2 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:42.160645 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc6c7a-10e8-4520-82e2-e8c504c781b2 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:42.160645 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc6c7a-10e8-4520-82e2-e8c504c781b2 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:42.162715 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 415/840] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:42.170687 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848465b-0fc3-4db1-969b-a3073da5de67 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:42.171124 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848465b-0fc3-4db1-969b-a3073da5de67 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:42.171237 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848465b-0fc3-4db1-969b-a3073da5de67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:42.190100 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-9848465b-0fc3-4db1-969b-a3073da5de67 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:42.191299 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-9848465b-0fc3-4db1-969b-a3073da5de67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:42.199327 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:42.267533 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 426/841] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:16:42.533348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc6c7a-10e8-4520-82e2-e8c504c781b2 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:42.642132 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc6c7a-10e8-4520-82e2-e8c504c781b2 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:42.678475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23746104-095a-4379-8f93-0d1023dd1861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:42.679121 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23746104-095a-4379-8f93-0d1023dd1861 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:42.679390 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23746104-095a-4379-8f93-0d1023dd1861 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:42.679649 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23746104-095a-4379-8f93-0d1023dd1861 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:42.680873 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23746104-095a-4379-8f93-0d1023dd1861 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:42.682294 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23746104-095a-4379-8f93-0d1023dd1861 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:42.683896 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 416/842] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:42.690095 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9848465b-0fc3-4db1-969b-a3073da5de67 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:42.796424 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9848465b-0fc3-4db1-969b-a3073da5de67 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:42.831005 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc84c9a-ee48-4903-8c18-c6ae442cbb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:42.831594 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc84c9a-ee48-4903-8c18-c6ae442cbb24 None admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:42.831860 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc84c9a-ee48-4903-8c18-c6ae442cbb24 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:42.832123 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc84c9a-ee48-4903-8c18-c6ae442cbb24 None admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:42.832773 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc84c9a-ee48-4903-8c18-c6ae442cbb24 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:42.834304 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc84c9a-ee48-4903-8c18-c6ae442cbb24 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:42.851956 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:42.858004 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 427/843] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 07:16:42 2022] POST /identity/v3/users => generated 286 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:45.013325 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f244b-2b44-4846-8461-73f7caef3e2c None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:45.014167 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f244b-2b44-4846-8461-73f7caef3e2c None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:45.014167 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f244b-2b44-4846-8461-73f7caef3e2c None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:45.016289 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 417/844] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:45.021975 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed196b-3abd-4f06-aa29-3368edd368ec None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:45.022399 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed196b-3abd-4f06-aa29-3368edd368ec None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:45.022488 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed196b-3abd-4f06-aa29-3368edd368ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:45.042468 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-1aed196b-3abd-4f06-aa29-3368edd368ec None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:45.043686 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-1aed196b-3abd-4f06-aa29-3368edd368ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:45.113461 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 428/845] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:45.380116 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f244b-2b44-4846-8461-73f7caef3e2c None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:45.463449 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:45.504366 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f244b-2b44-4846-8461-73f7caef3e2c None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:45.539657 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498237bf-ae60-462f-977e-579560b28377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:45.540272 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498237bf-ae60-462f-977e-579560b28377 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:45.540547 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498237bf-ae60-462f-977e-579560b28377 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:45.540816 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498237bf-ae60-462f-977e-579560b28377 None admin] PATH_INFO: `/v3/roles/39eab44f330440ebbefe39e538462165` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:45.542317 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498237bf-ae60-462f-977e-579560b28377 None admin] RBAC: Authorizing `identity:get_role(role_id=39eab44f330440ebbefe39e538462165)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:45.543719 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498237bf-ae60-462f-977e-579560b28377 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:45.545024 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 418/846] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:45 2022] GET /identity/v3/roles/39eab44f330440ebbefe39e538462165 => generated 223 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:45.551166 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed196b-3abd-4f06-aa29-3368edd368ec None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:45.657801 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed196b-3abd-4f06-aa29-3368edd368ec None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:45.685383 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:45.694282 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb330a0e-8bfc-4f72-b548-fbaacae59a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:45.694947 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb330a0e-8bfc-4f72-b548-fbaacae59a17 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:45.695219 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb330a0e-8bfc-4f72-b548-fbaacae59a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:45.695484 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb330a0e-8bfc-4f72-b548-fbaacae59a17 None admin] PATH_INFO: `/v3/users/b797d8a7fd55492cb6b02178a5b32515` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:45.708779 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb330a0e-8bfc-4f72-b548-fbaacae59a17 None admin] RBAC: Authorizing `identity:get_user(user_id=b797d8a7fd55492cb6b02178a5b32515)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:45.710229 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb330a0e-8bfc-4f72-b548-fbaacae59a17 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:45.711852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 429/847] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:45 2022] GET /identity/v3/users/b797d8a7fd55492cb6b02178a5b32515 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:45.718018 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498237bf-ae60-462f-977e-579560b28377 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:45.821547 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498237bf-ae60-462f-977e-579560b28377 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:45.850774 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:45.857871 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b52c65-4270-4cdd-a447-f111c83bf035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:45.858497 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b52c65-4270-4cdd-a447-f111c83bf035 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:45.858802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b52c65-4270-4cdd-a447-f111c83bf035 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:45.859070 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b52c65-4270-4cdd-a447-f111c83bf035 None admin] PATH_INFO: `/v3/projects/3b537ba94af04c8496b7251223333790` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:45.860235 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b52c65-4270-4cdd-a447-f111c83bf035 None admin] RBAC: Authorizing `identity:get_project(project_id=3b537ba94af04c8496b7251223333790)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:45.861663 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b52c65-4270-4cdd-a447-f111c83bf035 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:45.863347 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 419/848] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:45 2022] GET /identity/v3/projects/3b537ba94af04c8496b7251223333790 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:45.873106 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb330a0e-8bfc-4f72-b548-fbaacae59a17 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:45.982887 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb330a0e-8bfc-4f72-b548-fbaacae59a17 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:46.020118 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4244e5-c0c1-4580-8c8e-bffe8a903827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:46.020894 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4244e5-c0c1-4580-8c8e-bffe8a903827 None admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:46.021174 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4244e5-c0c1-4580-8c8e-bffe8a903827 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:46.021441 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4244e5-c0c1-4580-8c8e-bffe8a903827 None admin] PATH_INFO: `/v3/projects/3b537ba94af04c8496b7251223333790/users/b797d8a7fd55492cb6b02178a5b32515/roles/39eab44f330440ebbefe39e538462165` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:46.024257 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4244e5-c0c1-4580-8c8e-bffe8a903827 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b537ba94af04c8496b7251223333790, user_id=b797d8a7fd55492cb6b02178a5b32515, role_id=39eab44f330440ebbefe39e538462165)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:46.025695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4244e5-c0c1-4580-8c8e-bffe8a903827 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:46.045731 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 430/849] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 07:16:45 2022] PUT /identity/v3/projects/3b537ba94af04c8496b7251223333790/users/b797d8a7fd55492cb6b02178a5b32515/roles/39eab44f330440ebbefe39e538462165 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:16:48.177347 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95095773-debf-4370-aeca-576419f6aec3 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:48.177940 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95095773-debf-4370-aeca-576419f6aec3 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:48.177940 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95095773-debf-4370-aeca-576419f6aec3 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:48.179266 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 420/850] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:48.185622 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec766c60-9970-4790-a113-b84d63f4991c None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:48.186307 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec766c60-9970-4790-a113-b84d63f4991c None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:48.186499 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec766c60-9970-4790-a113-b84d63f4991c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:48.213838 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-ec766c60-9970-4790-a113-b84d63f4991c None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:48.215061 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-ec766c60-9970-4790-a113-b84d63f4991c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:48.258240 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:48.284180 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 431/851] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:48.562856 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95095773-debf-4370-aeca-576419f6aec3 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:48.672168 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95095773-debf-4370-aeca-576419f6aec3 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:48.707078 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b0f9b-308b-4aa5-a547-068bc32dd53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:48.707721 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b0f9b-308b-4aa5-a547-068bc32dd53b None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:48.707986 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b0f9b-308b-4aa5-a547-068bc32dd53b None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:48.708258 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b0f9b-308b-4aa5-a547-068bc32dd53b None admin] PATH_INFO: `/v3/roles/39eab44f330440ebbefe39e538462165` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:48.709738 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b0f9b-308b-4aa5-a547-068bc32dd53b None admin] RBAC: Authorizing `identity:get_role(role_id=39eab44f330440ebbefe39e538462165)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:48.711138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b0f9b-308b-4aa5-a547-068bc32dd53b None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:48.712455 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 421/852] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:48 2022] GET /identity/v3/roles/39eab44f330440ebbefe39e538462165 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:48.718608 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec766c60-9970-4790-a113-b84d63f4991c None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:48.821752 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec766c60-9970-4790-a113-b84d63f4991c None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:48.856031 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053ed2f-0b0f-4463-aadb-597bd73c5132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:48.856653 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053ed2f-0b0f-4463-aadb-597bd73c5132 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:48.856923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053ed2f-0b0f-4463-aadb-597bd73c5132 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:48.857186 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053ed2f-0b0f-4463-aadb-597bd73c5132 None admin] PATH_INFO: `/v3/users/b797d8a7fd55492cb6b02178a5b32515` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:48.858406 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5053ed2f-0b0f-4463-aadb-597bd73c5132 None admin] RBAC: Authorizing `identity:get_user(user_id=b797d8a7fd55492cb6b02178a5b32515)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:48.859875 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5053ed2f-0b0f-4463-aadb-597bd73c5132 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:48.861432 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 432/853] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:48 2022] GET /identity/v3/users/b797d8a7fd55492cb6b02178a5b32515 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:48.867576 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b0f9b-308b-4aa5-a547-068bc32dd53b None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:48.937063 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:48.975886 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b0f9b-308b-4aa5-a547-068bc32dd53b None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:49.011313 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e348be-d6a8-4f1a-9f4f-1fb8ac77e443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:49.011910 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e348be-d6a8-4f1a-9f4f-1fb8ac77e443 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:49.012181 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e348be-d6a8-4f1a-9f4f-1fb8ac77e443 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:49.012445 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e348be-d6a8-4f1a-9f4f-1fb8ac77e443 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:49.080739 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e348be-d6a8-4f1a-9f4f-1fb8ac77e443 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:49.082110 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e348be-d6a8-4f1a-9f4f-1fb8ac77e443 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:49.145808 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:49.184413 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 422/854] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 07:16:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:16:49.191398 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053ed2f-0b0f-4463-aadb-597bd73c5132 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:49.251519 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:49.309549 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053ed2f-0b0f-4463-aadb-597bd73c5132 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:49.345339 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186a400-68ae-4148-8585-e6c878a5ff1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:49.345897 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186a400-68ae-4148-8585-e6c878a5ff1f None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:49.346171 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186a400-68ae-4148-8585-e6c878a5ff1f None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:49.346434 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186a400-68ae-4148-8585-e6c878a5ff1f None admin] PATH_INFO: `/v3/projects/3b537ba94af04c8496b7251223333790` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:49.347615 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1186a400-68ae-4148-8585-e6c878a5ff1f None admin] RBAC: Authorizing `identity:get_project(project_id=3b537ba94af04c8496b7251223333790)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:49.349138 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1186a400-68ae-4148-8585-e6c878a5ff1f None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:49.350766 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 433/855] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:49 2022] GET /identity/v3/projects/3b537ba94af04c8496b7251223333790 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:49.357180 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e348be-d6a8-4f1a-9f4f-1fb8ac77e443 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:49.460863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e348be-d6a8-4f1a-9f4f-1fb8ac77e443 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:49.496080 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3699b3-6346-4a5d-95d4-3efcb50fec70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:49.496661 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3699b3-6346-4a5d-95d4-3efcb50fec70 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:49.496928 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3699b3-6346-4a5d-95d4-3efcb50fec70 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:49.497192 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3699b3-6346-4a5d-95d4-3efcb50fec70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:49.497900 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3699b3-6346-4a5d-95d4-3efcb50fec70 None admin] RBAC: Adding query filter params (role.id=39eab44f330440ebbefe39e538462165, scope.project.id=3b537ba94af04c8496b7251223333790, user.id=b797d8a7fd55492cb6b02178a5b32515) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:49.498206 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3699b3-6346-4a5d-95d4-3efcb50fec70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:49.499653 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3699b3-6346-4a5d-95d4-3efcb50fec70 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:49.505940 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 423/856] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 07:16:49 2022] GET /identity/v3/role_assignments?user.id=b797d8a7fd55492cb6b02178a5b32515&scope.project.id=3b537ba94af04c8496b7251223333790&role.id=39eab44f330440ebbefe39e538462165 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:51.662017 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e1036-5c7c-46a9-abb0-5ce5d7ed6010 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:51.662537 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e1036-5c7c-46a9-abb0-5ce5d7ed6010 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:51.662537 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e1036-5c7c-46a9-abb0-5ce5d7ed6010 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:51.664737 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 434/857] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:51.671350 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e4722-2249-419e-a645-710a1bb76a8e None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:51.672108 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e4722-2249-419e-a645-710a1bb76a8e None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:51.672230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e4722-2249-419e-a645-710a1bb76a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:51.698851 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-068e4722-2249-419e-a645-710a1bb76a8e None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:51.700040 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-068e4722-2249-419e-a645-710a1bb76a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:51.725259 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:51.770615 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 424/858] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:52.045658 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e1036-5c7c-46a9-abb0-5ce5d7ed6010 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:52.175838 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e1036-5c7c-46a9-abb0-5ce5d7ed6010 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:52.189199 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:52.211837 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950bc154-8cd7-4a81-982e-15db6773f714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:52.212452 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950bc154-8cd7-4a81-982e-15db6773f714 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:52.212726 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950bc154-8cd7-4a81-982e-15db6773f714 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:52.213001 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950bc154-8cd7-4a81-982e-15db6773f714 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:52.214138 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950bc154-8cd7-4a81-982e-15db6773f714 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:52.215609 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950bc154-8cd7-4a81-982e-15db6773f714 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:52.217172 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 435/859] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:52.225444 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068e4722-2249-419e-a645-710a1bb76a8e None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:52.325847 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068e4722-2249-419e-a645-710a1bb76a8e None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:52.361569 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4421784-963f-49cc-80ca-ce8d33fecee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:52.362156 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4421784-963f-49cc-80ca-ce8d33fecee2 None admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:52.362427 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4421784-963f-49cc-80ca-ce8d33fecee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:52.362701 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4421784-963f-49cc-80ca-ce8d33fecee2 None admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:52.363359 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4421784-963f-49cc-80ca-ce8d33fecee2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:52.364878 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4421784-963f-49cc-80ca-ce8d33fecee2 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:52.395665 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 425/860] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Nov 2 07:16:52 2022] POST /identity/v3/projects => generated 311 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:16:54.577500 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d229d-0e92-4895-9459-1fa42fca95bf None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:54.578376 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d229d-0e92-4895-9459-1fa42fca95bf None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:54.578527 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d229d-0e92-4895-9459-1fa42fca95bf None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:54.580630 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 436/861] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:54.587254 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b16cc61-c864-4e5d-9942-062af21139bd None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:54.587911 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b16cc61-c864-4e5d-9942-062af21139bd None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:54.588057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b16cc61-c864-4e5d-9942-062af21139bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:54.610094 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-7b16cc61-c864-4e5d-9942-062af21139bd None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:54.611320 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-7b16cc61-c864-4e5d-9942-062af21139bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:54.618832 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:54.681089 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 426/862] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:54.950627 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2d229d-0e92-4895-9459-1fa42fca95bf None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:55.066140 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2d229d-0e92-4895-9459-1fa42fca95bf None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:55.101144 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15923ff-c635-407c-93c3-a599ded4152e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:55.101762 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15923ff-c635-407c-93c3-a599ded4152e None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:55.102035 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15923ff-c635-407c-93c3-a599ded4152e None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:55.102305 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15923ff-c635-407c-93c3-a599ded4152e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:55.103443 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15923ff-c635-407c-93c3-a599ded4152e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:55.104885 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15923ff-c635-407c-93c3-a599ded4152e None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:55.106429 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 437/863] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:16:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:55.115007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b16cc61-c864-4e5d-9942-062af21139bd None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:55.215320 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b16cc61-c864-4e5d-9942-062af21139bd None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:55.253525 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91636011-aa50-4797-8cff-e5c0fca7daac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:55.254210 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91636011-aa50-4797-8cff-e5c0fca7daac None admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:55.254479 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91636011-aa50-4797-8cff-e5c0fca7daac None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:55.254854 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91636011-aa50-4797-8cff-e5c0fca7daac None admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:55.255493 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91636011-aa50-4797-8cff-e5c0fca7daac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:55.256923 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91636011-aa50-4797-8cff-e5c0fca7daac None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:55.277629 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:55.288775 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 427/864] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 07:16:55 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:16:57.451767 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c33dbe-5533-426e-bcbc-f9124edd6ab0 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:57.452359 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c33dbe-5533-426e-bcbc-f9124edd6ab0 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:57.452359 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c33dbe-5533-426e-bcbc-f9124edd6ab0 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:57.453642 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 438/865] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:16:57.460124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b79647d-99dc-4e4d-9aaa-7a54d43f324a None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:57.460801 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b79647d-99dc-4e4d-9aaa-7a54d43f324a None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:57.460973 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b79647d-99dc-4e4d-9aaa-7a54d43f324a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:57.484594 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-8b79647d-99dc-4e4d-9aaa-7a54d43f324a None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:16:57.485782 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-8b79647d-99dc-4e4d-9aaa-7a54d43f324a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:16:57.557204 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 428/866] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:16:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:16:57.824848 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c33dbe-5533-426e-bcbc-f9124edd6ab0 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:57.883547 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:57.945190 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c33dbe-5533-426e-bcbc-f9124edd6ab0 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:57.987895 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a77a10-8adb-4d31-b7a9-4737bcebbc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:57.988581 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a77a10-8adb-4d31-b7a9-4737bcebbc88 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:57.988851 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a77a10-8adb-4d31-b7a9-4737bcebbc88 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:57.989122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a77a10-8adb-4d31-b7a9-4737bcebbc88 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:57.999822 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a77a10-8adb-4d31-b7a9-4737bcebbc88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:16:58.004540 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-b3a77a10-8adb-4d31-b7a9-4737bcebbc88 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:58.004540 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:16:58.004540 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:16:58.004540 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:16:58.004540 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:16:58.004540 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:16:58.004540 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:16:58.004540 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:16:58.004540 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:16:58.004540 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:16:58.004540 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:16:58.004540 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:16:58.004540 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:16:58.004540 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:16:58.004540 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:16:58.004540 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:16:58.005585 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:16:58.005585 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:16:58.005585 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:16:58.005585 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:16:58.005585 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:16:58.005585 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:16:58.005585 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:16:58.005585 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:16:58.005585 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:16:58.005585 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:16:58.005585 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:16:58.005585 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:16:58.005585 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:16:58.005585 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:16:58.005585 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:16:58.005585 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:16:58.006793 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:16:58.006793 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:16:58.006793 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:16:58.006793 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:16:58.006793 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:16:58.006793 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:16:58.006793 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:16:58.006793 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:16:58.006793 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:16:58.006793 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:16:58.006793 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:16:58.006793 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:16:58.006793 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 439/867] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 07:16:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:16:58.014007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b79647d-99dc-4e4d-9aaa-7a54d43f324a None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:58.114928 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b79647d-99dc-4e4d-9aaa-7a54d43f324a None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:58.141363 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:58.155530 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff59e9d-bc40-4fd6-aa7d-7cee3b98210c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:58.156399 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff59e9d-bc40-4fd6-aa7d-7cee3b98210c None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:58.156639 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff59e9d-bc40-4fd6-aa7d-7cee3b98210c None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:58.156950 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff59e9d-bc40-4fd6-aa7d-7cee3b98210c None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:58.158003 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff59e9d-bc40-4fd6-aa7d-7cee3b98210c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:16:58.158446 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff59e9d-bc40-4fd6-aa7d-7cee3b98210c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:58.160810 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff59e9d-bc40-4fd6-aa7d-7cee3b98210c None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:58.169429 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 429/868] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:16:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:58.175671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a77a10-8adb-4d31-b7a9-4737bcebbc88 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:58.288653 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a77a10-8adb-4d31-b7a9-4737bcebbc88 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:58.298554 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:16:58.326006 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd879f3e-bd92-4cac-b6b9-3a0714529580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:58.326628 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd879f3e-bd92-4cac-b6b9-3a0714529580 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:58.326934 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd879f3e-bd92-4cac-b6b9-3a0714529580 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:58.327196 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd879f3e-bd92-4cac-b6b9-3a0714529580 None admin] PATH_INFO: `/v3/users/c191095a3f304f228ddddd04c0d2f490` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:58.340457 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd879f3e-bd92-4cac-b6b9-3a0714529580 None admin] RBAC: Authorizing `identity:get_user(user_id=c191095a3f304f228ddddd04c0d2f490)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:58.341893 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd879f3e-bd92-4cac-b6b9-3a0714529580 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:58.343511 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 440/869] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:16:58 2022] GET /identity/v3/users/c191095a3f304f228ddddd04c0d2f490 => generated 286 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:16:58.349927 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff59e9d-bc40-4fd6-aa7d-7cee3b98210c None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:58.453126 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff59e9d-bc40-4fd6-aa7d-7cee3b98210c None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:58.487920 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf1acf8-fd36-4cde-9f80-b08d8320ea22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:58.488735 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1acf8-fd36-4cde-9f80-b08d8320ea22 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:58.489010 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1acf8-fd36-4cde-9f80-b08d8320ea22 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:58.489277 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1acf8-fd36-4cde-9f80-b08d8320ea22 None admin] PATH_INFO: `/v3/projects/cb765d5adc104076a932105b6fd71949` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:58.490430 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf1acf8-fd36-4cde-9f80-b08d8320ea22 None admin] RBAC: Authorizing `identity:get_project(project_id=cb765d5adc104076a932105b6fd71949)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:58.491901 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf1acf8-fd36-4cde-9f80-b08d8320ea22 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:58.493528 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 430/870] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 07:16:58 2022] GET /identity/v3/projects/cb765d5adc104076a932105b6fd71949 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:16:58.499744 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd879f3e-bd92-4cac-b6b9-3a0714529580 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:16:58.618802 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd879f3e-bd92-4cac-b6b9-3a0714529580 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:16:58.655128 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7340ae22-d59d-419a-b746-a278f75cd8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:16:58.655780 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340ae22-d59d-419a-b746-a278f75cd8db None admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:16:58.656056 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340ae22-d59d-419a-b746-a278f75cd8db None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:16:58.656328 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340ae22-d59d-419a-b746-a278f75cd8db None admin] PATH_INFO: `/v3/projects/cb765d5adc104076a932105b6fd71949/users/c191095a3f304f228ddddd04c0d2f490/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:16:58.658487 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7340ae22-d59d-419a-b746-a278f75cd8db None admin] RBAC: Authorizing `identity:create_grant(project_id=cb765d5adc104076a932105b6fd71949, user_id=c191095a3f304f228ddddd04c0d2f490, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:16:58.659990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7340ae22-d59d-419a-b746-a278f75cd8db None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:16:58.676007 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 441/871] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 07:16:58 2022] PUT /identity/v3/projects/cb765d5adc104076a932105b6fd71949/users/c191095a3f304f228ddddd04c0d2f490/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:17:00.796496 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6745e6-60d0-4db8-bffc-d943f1e137be None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:00.797060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6745e6-60d0-4db8-bffc-d943f1e137be None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:00.797060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6745e6-60d0-4db8-bffc-d943f1e137be None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:00.798385 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 431/872] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:17:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:00.804991 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f535b25-f328-4cf8-b232-c921f2d9d177 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:00.805706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f535b25-f328-4cf8-b232-c921f2d9d177 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:00.805972 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f535b25-f328-4cf8-b232-c921f2d9d177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:00.834252 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-7f535b25-f328-4cf8-b232-c921f2d9d177 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:00.835482 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-7f535b25-f328-4cf8-b232-c921f2d9d177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:00.858789 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:00.911679 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 442/873] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:01.181887 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6745e6-60d0-4db8-bffc-d943f1e137be None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:01.247492 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:01.301017 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6745e6-60d0-4db8-bffc-d943f1e137be None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:01.336486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442398d3-eec5-4266-a216-328f3682c14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:01.337100 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442398d3-eec5-4266-a216-328f3682c14e None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:01.337375 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442398d3-eec5-4266-a216-328f3682c14e None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:01.337642 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442398d3-eec5-4266-a216-328f3682c14e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:01.348925 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442398d3-eec5-4266-a216-328f3682c14e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:17:01.353694 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-442398d3-eec5-4266-a216-328f3682c14e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:17:01.353694 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:01.353694 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:01.353694 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:01.353694 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:01.353694 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:01.353694 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:01.353694 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:01.353694 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:01.353694 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:01.353694 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:01.353694 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:01.353694 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:17:01.353694 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:17:01.353694 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:17:01.353694 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:17:01.354783 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:17:01.354783 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:17:01.354783 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:17:01.354783 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:17:01.354783 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:01.354783 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:01.354783 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:01.354783 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:01.354783 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:01.354783 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:01.354783 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:01.354783 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:01.354783 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:01.354783 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:01.354783 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:01.354783 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:01.355760 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:01.355760 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:01.355760 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:01.355760 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:01.355760 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:17:01.355760 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:17:01.355760 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:17:01.355760 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:17:01.355760 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:17:01.355760 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:17:01.355760 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:17:01.355760 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:17:01.355760 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 432/874] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 07:17:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:01.363179 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f535b25-f328-4cf8-b232-c921f2d9d177 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:01.472439 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f535b25-f328-4cf8-b232-c921f2d9d177 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:01.510244 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded6b23-70fe-4555-b616-f29639e41cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:01.510872 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded6b23-70fe-4555-b616-f29639e41cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:01.511145 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded6b23-70fe-4555-b616-f29639e41cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:01.511410 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded6b23-70fe-4555-b616-f29639e41cc0 None admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:01.512064 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded6b23-70fe-4555-b616-f29639e41cc0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:01.512360 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded6b23-70fe-4555-b616-f29639e41cc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:01.513734 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded6b23-70fe-4555-b616-f29639e41cc0 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:01.522126 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 443/875] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:17:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:01.528218 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442398d3-eec5-4266-a216-328f3682c14e None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:01.641238 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442398d3-eec5-4266-a216-328f3682c14e None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:01.659672 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:01.680839 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a67bf4-df46-4428-9978-bbe7648761c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:01.681508 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a67bf4-df46-4428-9978-bbe7648761c4 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:01.681790 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a67bf4-df46-4428-9978-bbe7648761c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:01.682059 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a67bf4-df46-4428-9978-bbe7648761c4 None admin] PATH_INFO: `/v3/users/c191095a3f304f228ddddd04c0d2f490` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:01.683225 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a67bf4-df46-4428-9978-bbe7648761c4 None admin] RBAC: Authorizing `identity:get_user(user_id=c191095a3f304f228ddddd04c0d2f490)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:01.684671 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a67bf4-df46-4428-9978-bbe7648761c4 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:01.686263 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 433/876] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:17:01 2022] GET /identity/v3/users/c191095a3f304f228ddddd04c0d2f490 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:01.692462 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded6b23-70fe-4555-b616-f29639e41cc0 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:01.712122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:01.794739 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded6b23-70fe-4555-b616-f29639e41cc0 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:01.829058 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec8de0-c130-490d-ae12-25f0a54ee774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:01.829711 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec8de0-c130-490d-ae12-25f0a54ee774 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:01.829917 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec8de0-c130-490d-ae12-25f0a54ee774 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:01.830187 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec8de0-c130-490d-ae12-25f0a54ee774 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:01.901229 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec8de0-c130-490d-ae12-25f0a54ee774 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:01.902688 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec8de0-c130-490d-ae12-25f0a54ee774 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:01.921287 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:02.007467 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 444/877] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 07:17:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:17:02.016916 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a67bf4-df46-4428-9978-bbe7648761c4 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:02.139771 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a67bf4-df46-4428-9978-bbe7648761c4 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:02.176618 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ad38c-adca-4d21-b9c5-f5f25fa84114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:02.177237 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ad38c-adca-4d21-b9c5-f5f25fa84114 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:02.177513 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ad38c-adca-4d21-b9c5-f5f25fa84114 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:02.177777 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ad38c-adca-4d21-b9c5-f5f25fa84114 None admin] PATH_INFO: `/v3/projects/cb765d5adc104076a932105b6fd71949` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:02.179026 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ad38c-adca-4d21-b9c5-f5f25fa84114 None admin] RBAC: Authorizing `identity:get_project(project_id=cb765d5adc104076a932105b6fd71949)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:02.180449 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ad38c-adca-4d21-b9c5-f5f25fa84114 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:02.182091 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 434/878] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 07:17:02 2022] GET /identity/v3/projects/cb765d5adc104076a932105b6fd71949 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:02.188660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec8de0-c130-490d-ae12-25f0a54ee774 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:02.296428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec8de0-c130-490d-ae12-25f0a54ee774 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:02.314726 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:02.335154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76455c25-557e-4b29-b22f-f25733cdcdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:02.335758 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76455c25-557e-4b29-b22f-f25733cdcdb7 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:02.336026 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76455c25-557e-4b29-b22f-f25733cdcdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:02.336284 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76455c25-557e-4b29-b22f-f25733cdcdb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:02.336992 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76455c25-557e-4b29-b22f-f25733cdcdb7 None admin] RBAC: Adding query filter params (role.id=1e35bb725eb8475ea63e2d9538ab67d5, scope.project.id=cb765d5adc104076a932105b6fd71949, user.id=c191095a3f304f228ddddd04c0d2f490) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:02.337377 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76455c25-557e-4b29-b22f-f25733cdcdb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:02.338851 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76455c25-557e-4b29-b22f-f25733cdcdb7 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:02.345247 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 445/879] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 07:17:02 2022] GET /identity/v3/role_assignments?user.id=c191095a3f304f228ddddd04c0d2f490&scope.project.id=cb765d5adc104076a932105b6fd71949&role.id=1e35bb725eb8475ea63e2d9538ab67d5 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:17:04.473661 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419e425-4ef2-4740-abac-e1e51d18aa6c None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:04.474520 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419e425-4ef2-4740-abac-e1e51d18aa6c None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:04.474520 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419e425-4ef2-4740-abac-e1e51d18aa6c None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:04.476636 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 435/880] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:17:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:04.482295 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87c149-0b5e-4bd1-8c79-888a2f39bfc6 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:04.482740 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87c149-0b5e-4bd1-8c79-888a2f39bfc6 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:04.482829 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87c149-0b5e-4bd1-8c79-888a2f39bfc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:04.502846 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-9a87c149-0b5e-4bd1-8c79-888a2f39bfc6 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:04.504083 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-9a87c149-0b5e-4bd1-8c79-888a2f39bfc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:04.578188 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 446/881] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:04.845227 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8419e425-4ef2-4740-abac-e1e51d18aa6c None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:04.907519 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:04.960580 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8419e425-4ef2-4740-abac-e1e51d18aa6c None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:05.000425 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf433e9-d13d-4d59-90d0-9e30cf824a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:05.001017 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf433e9-d13d-4d59-90d0-9e30cf824a7d None admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:05.001291 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf433e9-d13d-4d59-90d0-9e30cf824a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:05.001550 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf433e9-d13d-4d59-90d0-9e30cf824a7d None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:05.002211 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf433e9-d13d-4d59-90d0-9e30cf824a7d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:05.003896 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf433e9-d13d-4d59-90d0-9e30cf824a7d None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:05.028440 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 436/882] 10.222.0.30 () {60 vars in 1138 bytes} [Wed Nov 2 07:17:04 2022] POST /identity/v3/domains => generated 267 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:17:07.216773 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bca229-e551-4e15-a7a7-5f4228328ad0 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:07.217672 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bca229-e551-4e15-a7a7-5f4228328ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:07.217672 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bca229-e551-4e15-a7a7-5f4228328ad0 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:07.219797 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 447/883] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:17:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:07.229381 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00aa02c-54c5-43db-8dbb-85463d82942b None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:07.230000 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00aa02c-54c5-43db-8dbb-85463d82942b None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:07.230181 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00aa02c-54c5-43db-8dbb-85463d82942b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:07.258927 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-f00aa02c-54c5-43db-8dbb-85463d82942b None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:07.260101 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-f00aa02c-54c5-43db-8dbb-85463d82942b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:07.345905 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 437/884] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:07.611842 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bca229-e551-4e15-a7a7-5f4228328ad0 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:07.735921 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bca229-e551-4e15-a7a7-5f4228328ad0 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:07.749195 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:07.774332 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d2df7d-e28d-4a1e-843a-26997ea5ebb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:07.774999 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2df7d-e28d-4a1e-843a-26997ea5ebb3 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:07.775268 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2df7d-e28d-4a1e-843a-26997ea5ebb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:07.775531 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2df7d-e28d-4a1e-843a-26997ea5ebb3 None admin] PATH_INFO: `/v3/domains/1320c60e5b614463b447561c271d2e2c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:07.787193 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d2df7d-e28d-4a1e-843a-26997ea5ebb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1320c60e5b614463b447561c271d2e2c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:07.788660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d2df7d-e28d-4a1e-843a-26997ea5ebb3 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:07.790247 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 448/885] 10.222.0.30 () {56 vars in 1183 bytes} [Wed Nov 2 07:17:07 2022] GET /identity/v3/domains/1320c60e5b614463b447561c271d2e2c => generated 267 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:07.796365 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00aa02c-54c5-43db-8dbb-85463d82942b None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:07.844621 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:07.903662 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00aa02c-54c5-43db-8dbb-85463d82942b None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:07.941838 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2fe81e-4d94-4dc9-8694-f574b821344f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:07.942453 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fe81e-4d94-4dc9-8694-f574b821344f None admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:07.942716 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fe81e-4d94-4dc9-8694-f574b821344f None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:07.943013 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fe81e-4d94-4dc9-8694-f574b821344f None admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:07.943651 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2fe81e-4d94-4dc9-8694-f574b821344f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:07.945093 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2fe81e-4d94-4dc9-8694-f574b821344f None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:07.977664 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 438/886] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Nov 2 07:17:07 2022] POST /identity/v3/projects => generated 361 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:17:10.158963 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc92bf-687f-47ef-97e1-d2ebeebbd294 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:10.159914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc92bf-687f-47ef-97e1-d2ebeebbd294 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:10.159914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc92bf-687f-47ef-97e1-d2ebeebbd294 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:10.162066 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 449/887] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:17:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:10.169018 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fcf612-6854-49cc-a682-075ed6208bad None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:10.169479 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fcf612-6854-49cc-a682-075ed6208bad None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:10.169581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fcf612-6854-49cc-a682-075ed6208bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:10.194434 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-32fcf612-6854-49cc-a682-075ed6208bad None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:10.195660 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-32fcf612-6854-49cc-a682-075ed6208bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:10.270549 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 439/888] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:10.536688 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc92bf-687f-47ef-97e1-d2ebeebbd294 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:10.653519 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc92bf-687f-47ef-97e1-d2ebeebbd294 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:10.689451 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3e6b33-a002-4d75-a9a0-d28200d3c583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:10.690075 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e6b33-a002-4d75-a9a0-d28200d3c583 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:10.690408 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e6b33-a002-4d75-a9a0-d28200d3c583 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:10.690907 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e6b33-a002-4d75-a9a0-d28200d3c583 None admin] PATH_INFO: `/v3/domains/1320c60e5b614463b447561c271d2e2c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:10.692116 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3e6b33-a002-4d75-a9a0-d28200d3c583 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1320c60e5b614463b447561c271d2e2c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:10.693555 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3e6b33-a002-4d75-a9a0-d28200d3c583 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:10.695169 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 450/889] 10.222.0.30 () {56 vars in 1183 bytes} [Wed Nov 2 07:17:10 2022] GET /identity/v3/domains/1320c60e5b614463b447561c271d2e2c => generated 267 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:10.702691 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fcf612-6854-49cc-a682-075ed6208bad None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:10.727880 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:10.799501 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fcf612-6854-49cc-a682-075ed6208bad None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:10.833858 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a6e1f-4ecb-4689-b0ca-406d11e1bb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:10.834467 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a6e1f-4ecb-4689-b0ca-406d11e1bb4b None admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:10.834778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a6e1f-4ecb-4689-b0ca-406d11e1bb4b None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:10.835042 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a6e1f-4ecb-4689-b0ca-406d11e1bb4b None admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:10.835681 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a6e1f-4ecb-4689-b0ca-406d11e1bb4b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:10.837105 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a6e1f-4ecb-4689-b0ca-406d11e1bb4b None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:10.859631 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 440/890] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 07:17:10 2022] POST /identity/v3/users => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:17:12.994839 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f5023-e418-4dd1-a095-079bcad24c33 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:12.995726 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f5023-e418-4dd1-a095-079bcad24c33 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:12.995726 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f5023-e418-4dd1-a095-079bcad24c33 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:12.997736 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 451/891] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:17:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:13.003183 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5ae97-3e97-44b1-99cc-5317669a1149 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:13.003628 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5ae97-3e97-44b1-99cc-5317669a1149 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:13.003723 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5ae97-3e97-44b1-99cc-5317669a1149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:13.024298 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-5cb5ae97-3e97-44b1-99cc-5317669a1149 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:13.025467 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-5cb5ae97-3e97-44b1-99cc-5317669a1149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:13.097926 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 441/892] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:13.369741 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f5023-e418-4dd1-a095-079bcad24c33 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:13.431887 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:13.493393 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f5023-e418-4dd1-a095-079bcad24c33 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:13.530357 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79215200-67b1-42e9-9adc-c92c8095e386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:13.530996 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79215200-67b1-42e9-9adc-c92c8095e386 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:13.531266 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79215200-67b1-42e9-9adc-c92c8095e386 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:13.531529 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79215200-67b1-42e9-9adc-c92c8095e386 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:13.548764 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79215200-67b1-42e9-9adc-c92c8095e386 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:17:13.553814 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-79215200-67b1-42e9-9adc-c92c8095e386 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:17:13.553814 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:13.553814 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:13.553814 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:13.553814 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:13.553814 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:13.553814 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:13.553814 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:13.553814 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:13.553814 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:13.553814 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:13.553814 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:13.553814 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:17:13.553814 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:17:13.553814 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:17:13.553814 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:17:13.555053 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:17:13.555053 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:17:13.555053 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:17:13.555053 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:17:13.555053 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:13.555053 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:13.555053 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:13.555053 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:13.555053 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:13.555053 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:13.555053 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:13.555053 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:13.555053 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:13.555053 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:13.555053 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:13.555053 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:13.556280 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:13.556280 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:13.556280 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:13.556280 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:13.556280 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:17:13.556280 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:17:13.556280 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:17:13.556280 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:17:13.556280 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:17:13.556280 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:17:13.556280 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:17:13.556280 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:17:13.556280 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 452/893] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 07:17:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:13.561338 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5ae97-3e97-44b1-99cc-5317669a1149 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:13.575862 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:13.670439 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5ae97-3e97-44b1-99cc-5317669a1149 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:13.706165 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d665f6ad-063f-4f8c-b546-f184d9aaaa77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:13.706791 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d665f6ad-063f-4f8c-b546-f184d9aaaa77 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:13.707063 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d665f6ad-063f-4f8c-b546-f184d9aaaa77 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:13.707323 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d665f6ad-063f-4f8c-b546-f184d9aaaa77 None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:13.707971 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d665f6ad-063f-4f8c-b546-f184d9aaaa77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:13.708262 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d665f6ad-063f-4f8c-b546-f184d9aaaa77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:13.709626 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d665f6ad-063f-4f8c-b546-f184d9aaaa77 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:13.717596 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 442/894] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:17:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:13.726871 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79215200-67b1-42e9-9adc-c92c8095e386 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:13.828725 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:13.832040 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79215200-67b1-42e9-9adc-c92c8095e386 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:13.867229 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ec482-7a9b-4688-bb5a-39beea412a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:13.867848 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ec482-7a9b-4688-bb5a-39beea412a8d None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:13.868118 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ec482-7a9b-4688-bb5a-39beea412a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:13.868385 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ec482-7a9b-4688-bb5a-39beea412a8d None admin] PATH_INFO: `/v3/users/438b08e683874e5c89dc0c0a53ae11f4` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:13.881838 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ec482-7a9b-4688-bb5a-39beea412a8d None admin] RBAC: Authorizing `identity:get_user(user_id=438b08e683874e5c89dc0c0a53ae11f4)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:13.883325 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ec482-7a9b-4688-bb5a-39beea412a8d None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:13.884958 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 453/895] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:17:13 2022] GET /identity/v3/users/438b08e683874e5c89dc0c0a53ae11f4 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:13.892021 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d665f6ad-063f-4f8c-b546-f184d9aaaa77 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:13.970762 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:13.995967 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d665f6ad-063f-4f8c-b546-f184d9aaaa77 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:14.041787 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caf7d97-da8d-44b3-b44e-df8c3be593e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:14.042391 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf7d97-da8d-44b3-b44e-df8c3be593e8 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:14.042701 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf7d97-da8d-44b3-b44e-df8c3be593e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:14.042976 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf7d97-da8d-44b3-b44e-df8c3be593e8 None admin] PATH_INFO: `/v3/projects/561a8dcdcc3a435085aaeb0a32d35cd1` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:14.044158 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caf7d97-da8d-44b3-b44e-df8c3be593e8 None admin] RBAC: Authorizing `identity:get_project(project_id=561a8dcdcc3a435085aaeb0a32d35cd1)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:14.045599 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caf7d97-da8d-44b3-b44e-df8c3be593e8 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:14.047273 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 443/896] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 07:17:13 2022] GET /identity/v3/projects/561a8dcdcc3a435085aaeb0a32d35cd1 => generated 361 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:14.054134 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ec482-7a9b-4688-bb5a-39beea412a8d None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:14.170650 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ec482-7a9b-4688-bb5a-39beea412a8d None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:14.209254 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac6874-a8c0-4600-8f04-76f8624af842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:14.209903 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac6874-a8c0-4600-8f04-76f8624af842 None admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:14.210177 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac6874-a8c0-4600-8f04-76f8624af842 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:14.210445 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac6874-a8c0-4600-8f04-76f8624af842 None admin] PATH_INFO: `/v3/projects/561a8dcdcc3a435085aaeb0a32d35cd1/users/438b08e683874e5c89dc0c0a53ae11f4/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:14.212446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac6874-a8c0-4600-8f04-76f8624af842 None admin] RBAC: Authorizing `identity:create_grant(project_id=561a8dcdcc3a435085aaeb0a32d35cd1, user_id=438b08e683874e5c89dc0c0a53ae11f4, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:14.213888 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac6874-a8c0-4600-8f04-76f8624af842 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:14.234937 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 454/897] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 07:17:14 2022] PUT /identity/v3/projects/561a8dcdcc3a435085aaeb0a32d35cd1/users/438b08e683874e5c89dc0c0a53ae11f4/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:17:16.356141 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2c9a4-64f8-4a50-a0ba-529873590787 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:16.356715 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2c9a4-64f8-4a50-a0ba-529873590787 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:16.356715 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2c9a4-64f8-4a50-a0ba-529873590787 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:16.358731 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 444/898] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:17:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:17:16.367637 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3877e-6503-4ed2-b7e8-cf011455cd85 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:16.368043 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3877e-6503-4ed2-b7e8-cf011455cd85 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:16.368228 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3877e-6503-4ed2-b7e8-cf011455cd85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:16.391018 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-b7a3877e-6503-4ed2-b7e8-cf011455cd85 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:17:16.392218 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-b7a3877e-6503-4ed2-b7e8-cf011455cd85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:17:16.406575 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:16.472037 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 455/899] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:17:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:17:16.732429 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d2c9a4-64f8-4a50-a0ba-529873590787 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:16.845560 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d2c9a4-64f8-4a50-a0ba-529873590787 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:16.879537 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31815976-f321-4b9b-81dd-f11d995639e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:16.880089 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31815976-f321-4b9b-81dd-f11d995639e6 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:16.880364 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31815976-f321-4b9b-81dd-f11d995639e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:16.880631 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31815976-f321-4b9b-81dd-f11d995639e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:16.891876 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31815976-f321-4b9b-81dd-f11d995639e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:17:16.896985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-31815976-f321-4b9b-81dd-f11d995639e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:17:16.896985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:17:16.896985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:17:16.896985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:17:16.896985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:17:16.896985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:17:16.896985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:17:16.896985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:17:16.896985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:17:16.896985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:17:16.896985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:17:16.896985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:17:16.896985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:17:16.896985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:17:16.896985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:17:16.896985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:17:16.898073 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:17:16.898073 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:17:16.898073 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:17:16.898073 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:17:16.898073 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:17:16.898073 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:17:16.898073 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:17:16.898073 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:17:16.898073 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:17:16.898073 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:17:16.898073 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:17:16.898073 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:17:16.898073 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:17:16.898073 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:17:16.898073 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:17:16.898073 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:17:16.899033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:17:16.899033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:17:16.899033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:17:16.899033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:17:16.899033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:17:16.899033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:17:16.899033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:17:16.899033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:17:16.899033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:17:16.899033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:17:16.899033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:17:16.899033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:17:16.899033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 445/900] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 07:17:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:17:16.905015 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a3877e-6503-4ed2-b7e8-cf011455cd85 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:17.012550 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a3877e-6503-4ed2-b7e8-cf011455cd85 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:17.047472 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3e1e54-c91d-4788-b43d-9d5b5dc96f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:17.048066 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e1e54-c91d-4788-b43d-9d5b5dc96f0c None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:17.048340 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e1e54-c91d-4788-b43d-9d5b5dc96f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:17.048608 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e1e54-c91d-4788-b43d-9d5b5dc96f0c None admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:17.049295 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3e1e54-c91d-4788-b43d-9d5b5dc96f0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:17.049578 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3e1e54-c91d-4788-b43d-9d5b5dc96f0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:17.051073 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3e1e54-c91d-4788-b43d-9d5b5dc96f0c None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:17.059182 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 456/901] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:17:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:17.065808 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31815976-f321-4b9b-81dd-f11d995639e6 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:17.068840 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:17.171871 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31815976-f321-4b9b-81dd-f11d995639e6 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:17.206571 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9072b0-674a-4165-a4e4-e0d3398488c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:17.207215 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9072b0-674a-4165-a4e4-e0d3398488c6 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:17.207489 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9072b0-674a-4165-a4e4-e0d3398488c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:17.207725 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9072b0-674a-4165-a4e4-e0d3398488c6 None admin] PATH_INFO: `/v3/users/438b08e683874e5c89dc0c0a53ae11f4` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:17.208965 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9072b0-674a-4165-a4e4-e0d3398488c6 None admin] RBAC: Authorizing `identity:get_user(user_id=438b08e683874e5c89dc0c0a53ae11f4)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:17.210389 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9072b0-674a-4165-a4e4-e0d3398488c6 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:17.211984 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 446/902] 10.222.0.30 () {56 vars in 1177 bytes} [Wed Nov 2 07:17:17 2022] GET /identity/v3/users/438b08e683874e5c89dc0c0a53ae11f4 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:17.217988 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3e1e54-c91d-4788-b43d-9d5b5dc96f0c None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:17.231185 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:17.322439 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3e1e54-c91d-4788-b43d-9d5b5dc96f0c None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:17.360873 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5ae1d-4d01-4c1d-9abb-4e1ce05d7aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:17.361470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5ae1d-4d01-4c1d-9abb-4e1ce05d7aa0 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:17.361777 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5ae1d-4d01-4c1d-9abb-4e1ce05d7aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:17.362052 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5ae1d-4d01-4c1d-9abb-4e1ce05d7aa0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:17.438197 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c5ae1d-4d01-4c1d-9abb-4e1ce05d7aa0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:17.439613 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c5ae1d-4d01-4c1d-9abb-4e1ce05d7aa0 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:17.450485 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:17.566689 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 457/903] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 07:17:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:17:17.573064 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9072b0-674a-4165-a4e4-e0d3398488c6 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:17.656146 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:17.690017 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9072b0-674a-4165-a4e4-e0d3398488c6 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:17.731007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa2a44e-00b9-4f1f-b6bd-3b69598645c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:17.731624 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2a44e-00b9-4f1f-b6bd-3b69598645c9 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:17.731893 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2a44e-00b9-4f1f-b6bd-3b69598645c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:17.732155 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2a44e-00b9-4f1f-b6bd-3b69598645c9 None admin] PATH_INFO: `/v3/projects/561a8dcdcc3a435085aaeb0a32d35cd1` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:17.733479 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa2a44e-00b9-4f1f-b6bd-3b69598645c9 None admin] RBAC: Authorizing `identity:get_project(project_id=561a8dcdcc3a435085aaeb0a32d35cd1)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:17.734950 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa2a44e-00b9-4f1f-b6bd-3b69598645c9 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:17.736550 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 447/904] 10.222.0.30 () {56 vars in 1186 bytes} [Wed Nov 2 07:17:17 2022] GET /identity/v3/projects/561a8dcdcc3a435085aaeb0a32d35cd1 => generated 361 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:17.743780 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5ae1d-4d01-4c1d-9abb-4e1ce05d7aa0 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:17:17.848181 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5ae1d-4d01-4c1d-9abb-4e1ce05d7aa0 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:17:17.858580 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:17.889519 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4391ae-15f7-4fcc-89bb-cc8a1a84f690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:17:17.890123 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4391ae-15f7-4fcc-89bb-cc8a1a84f690 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:17:17.890392 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4391ae-15f7-4fcc-89bb-cc8a1a84f690 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:17:17.890643 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4391ae-15f7-4fcc-89bb-cc8a1a84f690 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:17:17.891392 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4391ae-15f7-4fcc-89bb-cc8a1a84f690 None admin] RBAC: Adding query filter params (role.id=1e35bb725eb8475ea63e2d9538ab67d5, scope.project.id=561a8dcdcc3a435085aaeb0a32d35cd1, user.id=438b08e683874e5c89dc0c0a53ae11f4) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:17:17.891695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4391ae-15f7-4fcc-89bb-cc8a1a84f690 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:17:17.893406 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4391ae-15f7-4fcc-89bb-cc8a1a84f690 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:17:17.899722 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 458/905] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 07:17:17 2022] GET /identity/v3/role_assignments?user.id=438b08e683874e5c89dc0c0a53ae11f4&scope.project.id=561a8dcdcc3a435085aaeb0a32d35cd1&role.id=1e35bb725eb8475ea63e2d9538ab67d5 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:17:27.724535 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=31 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:17:27.897150 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=13 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:14.024005 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d27d5-0246-4de2-9305-d441a2f13d84 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:14.024639 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d27d5-0246-4de2-9305-d441a2f13d84 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:14.024639 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d27d5-0246-4de2-9305-d441a2f13d84 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:14.026015 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 448/906] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:14.036749 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49f91e-8b49-45a0-ba9d-3d053138fdaa None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:14.037319 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49f91e-8b49-45a0-ba9d-3d053138fdaa None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:14.037319 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49f91e-8b49-45a0-ba9d-3d053138fdaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:14.059252 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-0f49f91e-8b49-45a0-ba9d-3d053138fdaa None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:14.059481 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-0f49f91e-8b49-45a0-ba9d-3d053138fdaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:14.146999 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 459/907] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:18:14.426251 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d27d5-0246-4de2-9305-d441a2f13d84 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:14.549706 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d27d5-0246-4de2-9305-d441a2f13d84 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:14.595588 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0622804-119a-4035-b413-8ca61a741867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:14.596269 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0622804-119a-4035-b413-8ca61a741867 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:14.596675 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0622804-119a-4035-b413-8ca61a741867 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:14.597147 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0622804-119a-4035-b413-8ca61a741867 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:14.611629 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0622804-119a-4035-b413-8ca61a741867 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:14.613362 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0622804-119a-4035-b413-8ca61a741867 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:14.631166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-d0622804-119a-4035-b413-8ca61a741867 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:14.631166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:14.631166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:14.631166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:14.631166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:14.631166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:14.631166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:14.631166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:14.631166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:14.631166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:14.631166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:14.631166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:14.631166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:18:14.631166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:18:14.631166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:18:14.631166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:18:14.633037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:14.633037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:14.633037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:14.633037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:14.633037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:14.633037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:14.633037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:14.633037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:14.633037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:14.633037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:14.633037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:14.633037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:14.633037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:14.633037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:14.633037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:14.633037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:14.633037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:18:14.634001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:18:14.634001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:14.634001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:18:14.634001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 449/908] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:14.646129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49f91e-8b49-45a0-ba9d-3d053138fdaa None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:14.785935 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49f91e-8b49-45a0-ba9d-3d053138fdaa None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:14.825168 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:14.832642 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ffdcd-894b-4938-a322-f6685720d779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:14.833301 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ffdcd-894b-4938-a322-f6685720d779 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:14.833614 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ffdcd-894b-4938-a322-f6685720d779 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:14.833908 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ffdcd-894b-4938-a322-f6685720d779 None admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:14.834610 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150ffdcd-894b-4938-a322-f6685720d779 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:14.835155 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150ffdcd-894b-4938-a322-f6685720d779 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:14.836847 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150ffdcd-894b-4938-a322-f6685720d779 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:14.850728 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 460/909] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:14.856354 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0622804-119a-4035-b413-8ca61a741867 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:14.931995 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:14.996594 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0622804-119a-4035-b413-8ca61a741867 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:15.048149 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f305bf-6bba-4c65-9395-439a8a9c51d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:15.049050 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f305bf-6bba-4c65-9395-439a8a9c51d7 None admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:15.049262 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f305bf-6bba-4c65-9395-439a8a9c51d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:15.049659 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f305bf-6bba-4c65-9395-439a8a9c51d7 None admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:15.050461 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f305bf-6bba-4c65-9395-439a8a9c51d7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:15.052219 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f305bf-6bba-4c65-9395-439a8a9c51d7 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:15.080213 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 450/910] 10.222.0.30 () {60 vars in 1132 bytes} [Wed Nov 2 07:18:14 2022] POST /identity/v3/users => generated 251 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:18:17.397593 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ee091-1705-48f0-8928-209d3bf6fdea None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:17.398203 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ee091-1705-48f0-8928-209d3bf6fdea None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:17.398203 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ee091-1705-48f0-8928-209d3bf6fdea None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:17.399527 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 461/911] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:17.405107 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca539016-af3f-4430-ab3f-2f98d4ff8e2e None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:17.405563 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca539016-af3f-4430-ab3f-2f98d4ff8e2e None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:17.405677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca539016-af3f-4430-ab3f-2f98d4ff8e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:17.426226 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-ca539016-af3f-4430-ab3f-2f98d4ff8e2e None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:17.427468 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-ca539016-af3f-4430-ab3f-2f98d4ff8e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:17.502427 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 451/912] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:18:17.766039 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ee091-1705-48f0-8928-209d3bf6fdea None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:17.896432 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ee091-1705-48f0-8928-209d3bf6fdea None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:17.940425 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33aae2b-5a8d-4a3a-beff-96e45e7164e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:17.941209 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33aae2b-5a8d-4a3a-beff-96e45e7164e9 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:17.941520 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33aae2b-5a8d-4a3a-beff-96e45e7164e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:17.941760 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33aae2b-5a8d-4a3a-beff-96e45e7164e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:17.952889 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33aae2b-5a8d-4a3a-beff-96e45e7164e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:18:17.957721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-a33aae2b-5a8d-4a3a-beff-96e45e7164e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:18:17.957721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:17.957721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:17.957721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:17.957721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:17.957721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:17.957721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:17.957721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:17.957721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:17.957721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:17.957721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:17.957721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:17.957721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:18:17.957721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:18:17.957721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:18:17.957721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:18:17.958950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:18:17.958950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:18:17.958950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:18:17.958950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:18:17.958950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:17.958950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:17.958950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:17.958950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:17.958950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:17.958950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:17.958950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:17.958950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:17.958950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:17.958950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:17.958950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:17.958950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:17.959820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:17.959820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:17.959820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:17.959820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:17.959820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:18:17.959820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:18:17.959820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:18:17.959820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:18:17.959820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:18:17.959820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:18:17.959820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:18:17.959820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:18:17.959820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 462/913] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Nov 2 07:18:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:17.965660 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca539016-af3f-4430-ab3f-2f98d4ff8e2e None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:18.016544 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:18.083998 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca539016-af3f-4430-ab3f-2f98d4ff8e2e None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:18.124198 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655944a-c506-41e9-8ad2-b4edc21ce25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:18.124924 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655944a-c506-41e9-8ad2-b4edc21ce25d None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:18.125197 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655944a-c506-41e9-8ad2-b4edc21ce25d None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:18.125490 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655944a-c506-41e9-8ad2-b4edc21ce25d None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:18.126214 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3655944a-c506-41e9-8ad2-b4edc21ce25d None admin] RBAC: Adding query filter params (name=service) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:18.126547 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3655944a-c506-41e9-8ad2-b4edc21ce25d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:18.128856 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3655944a-c506-41e9-8ad2-b4edc21ce25d None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:18.137426 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 452/914] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Nov 2 07:18:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:18.144960 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33aae2b-5a8d-4a3a-beff-96e45e7164e9 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:18.198905 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:18.263000 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33aae2b-5a8d-4a3a-beff-96e45e7164e9 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:18.302475 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44170723-8757-4702-9cfe-4b4507ae47f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:18.303045 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44170723-8757-4702-9cfe-4b4507ae47f7 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:18.303316 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44170723-8757-4702-9cfe-4b4507ae47f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:18.303583 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44170723-8757-4702-9cfe-4b4507ae47f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:18.317278 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44170723-8757-4702-9cfe-4b4507ae47f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:18.319522 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44170723-8757-4702-9cfe-4b4507ae47f7 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:18.333886 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-44170723-8757-4702-9cfe-4b4507ae47f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:18.333886 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:18.333886 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:18.333886 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:18.333886 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:18.333886 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:18.333886 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:18.333886 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:18.333886 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:18.333886 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:18.333886 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:18.333886 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:18.333886 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:18:18.333886 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:18:18.333886 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:18:18.333886 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:18:18.335099 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:18.335099 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:18.335099 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:18.335099 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:18.335099 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:18.335099 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:18.335099 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:18.335099 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:18.335099 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:18.335099 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:18.335099 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:18.335099 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:18.335099 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:18.335099 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:18.335099 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:18.335099 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:18.335099 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:18:18.340967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:18:18.340967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:18.340967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:18:18.340967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 463/915] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:18.341251 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655944a-c506-41e9-8ad2-b4edc21ce25d None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:18.450980 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:18.463251 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655944a-c506-41e9-8ad2-b4edc21ce25d None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:18.499025 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9340b-6824-4759-97f3-51263aeeeea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:18.499900 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9340b-6824-4759-97f3-51263aeeeea2 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:18.500346 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9340b-6824-4759-97f3-51263aeeeea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:18.500798 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9340b-6824-4759-97f3-51263aeeeea2 None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:18.501579 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9340b-6824-4759-97f3-51263aeeeea2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:18.502074 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9340b-6824-4759-97f3-51263aeeeea2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:18.503643 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9340b-6824-4759-97f3-51263aeeeea2 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:18.523332 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 453/916] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:18:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:18.524497 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44170723-8757-4702-9cfe-4b4507ae47f7 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:18.640447 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44170723-8757-4702-9cfe-4b4507ae47f7 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:18.642105 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:18.678876 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e861363-b30b-41ba-9265-87ae83bfaec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:18.679617 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e861363-b30b-41ba-9265-87ae83bfaec3 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:18.679967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e861363-b30b-41ba-9265-87ae83bfaec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:18.680326 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e861363-b30b-41ba-9265-87ae83bfaec3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:18.688294 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e861363-b30b-41ba-9265-87ae83bfaec3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:18.689960 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e861363-b30b-41ba-9265-87ae83bfaec3 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:18.701672 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-6e861363-b30b-41ba-9265-87ae83bfaec3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Nov 02 07:18:18.701672 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:18.701672 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:18.701672 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:18.701672 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:18.701672 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:18.701672 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:18.701672 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:18.701672 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:18.701672 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:18.701672 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:18.701672 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:18.701672 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:18:18.701672 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:18:18.701672 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:18:18.701672 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:18:18.702923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:18.702923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:18.702923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:18:18.702923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:18:18.702923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:18:18.702923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:18:18.702923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:18.702923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:18.702923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:18.702923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:18.702923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:18.702923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:18.702923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:18.702923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:18.702923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:18.702923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:18.702923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:18.702923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:18.704145 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:18.704145 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:18.704145 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:18:18.704145 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:18:18.704145 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:18:18.704145 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:18:18.704145 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:18:18.704145 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:18:18.704145 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Nov 02 07:18:18.704145 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:18:18.711488 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 464/917] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:18.711629 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9340b-6824-4759-97f3-51263aeeeea2 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:18.828397 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9340b-6824-4759-97f3-51263aeeeea2 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:18.864114 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e0088-013b-433c-b159-16ee8c4f0267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:18.864662 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e0088-013b-433c-b159-16ee8c4f0267 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:18.864982 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e0088-013b-433c-b159-16ee8c4f0267 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:18.865252 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e0088-013b-433c-b159-16ee8c4f0267 None admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:18.866019 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4e0088-013b-433c-b159-16ee8c4f0267 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:18.866327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4e0088-013b-433c-b159-16ee8c4f0267 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:18.867831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4e0088-013b-433c-b159-16ee8c4f0267 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:18.893132 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 454/918] 10.222.0.30 () {56 vars in 1143 bytes} [Wed Nov 2 07:18:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:18:18.895194 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e861363-b30b-41ba-9265-87ae83bfaec3 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:19.015797 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e861363-b30b-41ba-9265-87ae83bfaec3 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:19.060174 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7789c0-367f-421a-a8cf-56881789dad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:19.061180 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7789c0-367f-421a-a8cf-56881789dad8 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:19.061662 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7789c0-367f-421a-a8cf-56881789dad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:19.061920 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7789c0-367f-421a-a8cf-56881789dad8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:19.073781 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7789c0-367f-421a-a8cf-56881789dad8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:19.075270 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7789c0-367f-421a-a8cf-56881789dad8 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:19.090577 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-6d7789c0-367f-421a-a8cf-56881789dad8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:19.090577 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:19.090577 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:19.090577 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:19.090577 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:19.090577 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:19.090577 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:19.090577 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:19.090577 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:19.090577 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:19.090577 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:19.090577 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:19.090577 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:18:19.090577 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:18:19.090577 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:18:19.090577 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:18:19.091648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:19.091648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:19.091648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:19.091648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:19.091648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:19.091648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:19.091648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:19.091648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:19.091648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:19.091648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:19.091648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:19.091648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:19.091648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:19.091648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:19.091648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:19.091648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:19.091648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:18:19.092541 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:18:19.092541 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:19.092541 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:18:19.092541 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 465/919] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:19.099918 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e0088-013b-433c-b159-16ee8c4f0267 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:19.121892 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:19.215505 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e0088-013b-433c-b159-16ee8c4f0267 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:19.256322 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6bf7b-fa9b-4409-b7be-f76ed1b3858d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:19.257058 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6bf7b-fa9b-4409-b7be-f76ed1b3858d None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:19.257334 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6bf7b-fa9b-4409-b7be-f76ed1b3858d None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:19.257635 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6bf7b-fa9b-4409-b7be-f76ed1b3858d None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:19.258344 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6bf7b-fa9b-4409-b7be-f76ed1b3858d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:19.258745 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6bf7b-fa9b-4409-b7be-f76ed1b3858d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:19.260425 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6bf7b-fa9b-4409-b7be-f76ed1b3858d None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:19.274833 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 455/920] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:18:19.280884 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7789c0-367f-421a-a8cf-56881789dad8 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:19.300773 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:19.392448 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7789c0-367f-421a-a8cf-56881789dad8 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:19.435762 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff67f5e-f385-44c9-a88d-7e0eaef848c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:19.436420 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff67f5e-f385-44c9-a88d-7e0eaef848c3 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:19.436674 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff67f5e-f385-44c9-a88d-7e0eaef848c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:19.436943 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff67f5e-f385-44c9-a88d-7e0eaef848c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:19.444434 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff67f5e-f385-44c9-a88d-7e0eaef848c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:19.445969 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff67f5e-f385-44c9-a88d-7e0eaef848c3 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:19.458929 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-6ff67f5e-f385-44c9-a88d-7e0eaef848c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:18:19.458929 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:19.458929 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:19.458929 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:19.458929 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:19.458929 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:19.458929 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:19.458929 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:19.458929 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:19.458929 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:19.458929 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:19.458929 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:19.458929 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:18:19.458929 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:18:19.458929 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:18:19.460074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:18:19.460074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:19.460074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:19.460074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:19.460074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:19.460074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:19.460074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:19.460074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:19.460074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:19.460074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:19.460074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:19.460074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:19.460074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:19.460074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:19.460074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:19.460074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:19.460074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:19.460985 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:18:19.460985 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:18:19.460985 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:18:19.460985 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:18:19.460985 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:18:19.460985 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:18:19.460985 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:18:19.460985 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:18:19.460985 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 466/921] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:18:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:19.466203 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6bf7b-fa9b-4409-b7be-f76ed1b3858d None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:19.563853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:19.587010 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6bf7b-fa9b-4409-b7be-f76ed1b3858d None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:19.626962 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11544b6-d395-4415-89df-2eae0d3b634b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:19.627744 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11544b6-d395-4415-89df-2eae0d3b634b None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:19.628045 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11544b6-d395-4415-89df-2eae0d3b634b None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:19.628291 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11544b6-d395-4415-89df-2eae0d3b634b None admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:19.628989 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11544b6-d395-4415-89df-2eae0d3b634b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:19.629293 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11544b6-d395-4415-89df-2eae0d3b634b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:19.630861 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11544b6-d395-4415-89df-2eae0d3b634b None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:19.645151 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 456/922] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 07:18:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:19.651511 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff67f5e-f385-44c9-a88d-7e0eaef848c3 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:19.737911 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:19.768385 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff67f5e-f385-44c9-a88d-7e0eaef848c3 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:19.803292 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e2a634-098d-4f9c-969d-973c0987a394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:19.803959 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2a634-098d-4f9c-969d-973c0987a394 None admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:19.804244 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2a634-098d-4f9c-969d-973c0987a394 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:19.806334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2a634-098d-4f9c-969d-973c0987a394 None admin] PATH_INFO: `/v3/projects/25e2bc8417754f909b416b93d5b7afb7/users/932d3076fb4b426d8c83d02eb60d8063/roles/268fecadd98446f2b7d41755f3fe3649` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:19.821956 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e2a634-098d-4f9c-969d-973c0987a394 None admin] RBAC: Authorizing `identity:create_grant(project_id=25e2bc8417754f909b416b93d5b7afb7, user_id=932d3076fb4b426d8c83d02eb60d8063, role_id=268fecadd98446f2b7d41755f3fe3649)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:19.823453 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e2a634-098d-4f9c-969d-973c0987a394 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:19.836135 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 467/923] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 07:18:19 2022] PUT /identity/v3/projects/25e2bc8417754f909b416b93d5b7afb7/users/932d3076fb4b426d8c83d02eb60d8063/roles/268fecadd98446f2b7d41755f3fe3649 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:18:22.072282 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513dd31-162f-4912-80bd-c986470a8a35 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:22.072827 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513dd31-162f-4912-80bd-c986470a8a35 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:22.072827 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513dd31-162f-4912-80bd-c986470a8a35 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:22.074108 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 457/924] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:22.079702 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228f207-01ac-430a-96a3-8fbc04440cbd None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:22.080410 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228f207-01ac-430a-96a3-8fbc04440cbd None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:22.080410 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228f207-01ac-430a-96a3-8fbc04440cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:22.099108 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-a228f207-01ac-430a-96a3-8fbc04440cbd None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:22.100298 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-a228f207-01ac-430a-96a3-8fbc04440cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:22.172492 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 468/925] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:18:22.472400 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513dd31-162f-4912-80bd-c986470a8a35 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:22.583590 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513dd31-162f-4912-80bd-c986470a8a35 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:22.618848 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89360b33-5c55-4278-96df-57d89d3d6958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:22.619469 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89360b33-5c55-4278-96df-57d89d3d6958 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:22.619737 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89360b33-5c55-4278-96df-57d89d3d6958 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:22.620005 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89360b33-5c55-4278-96df-57d89d3d6958 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:22.627079 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:22.631217 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89360b33-5c55-4278-96df-57d89d3d6958 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:18:22.636204 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-89360b33-5c55-4278-96df-57d89d3d6958 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:18:22.636204 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:22.636204 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:22.636204 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:22.636204 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:22.636204 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:22.636204 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:22.636204 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:22.636204 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:22.636204 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:22.636204 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:22.636204 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:22.636204 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:18:22.636204 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:18:22.636204 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:18:22.636204 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:18:22.637348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:18:22.637348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:18:22.637348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:18:22.637348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:18:22.637348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:22.637348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:22.637348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:22.637348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:22.637348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:22.637348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:22.637348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:22.637348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:22.637348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:22.637348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:22.637348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:22.637348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:22.638551 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:22.638551 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:22.638551 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:22.638551 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:22.638551 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:18:22.638551 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:18:22.638551 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:18:22.638551 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:18:22.638551 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:18:22.638551 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:18:22.638551 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 07:18:22.638551 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:18:22.638551 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 458/926] 10.222.0.30 () {56 vars in 1102 bytes} [Wed Nov 2 07:18:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:22.643764 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a228f207-01ac-430a-96a3-8fbc04440cbd None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:22.702509 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:22.749139 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a228f207-01ac-430a-96a3-8fbc04440cbd None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:22.787927 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e28e3f-8c71-4aaa-a1ae-a75f90ba9eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:22.788583 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e28e3f-8c71-4aaa-a1ae-a75f90ba9eea None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:22.788862 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e28e3f-8c71-4aaa-a1ae-a75f90ba9eea None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:22.789122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e28e3f-8c71-4aaa-a1ae-a75f90ba9eea None admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:22.789774 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e28e3f-8c71-4aaa-a1ae-a75f90ba9eea None admin] RBAC: Adding query filter params (name=service) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:22.790067 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e28e3f-8c71-4aaa-a1ae-a75f90ba9eea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:22.791474 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e28e3f-8c71-4aaa-a1ae-a75f90ba9eea None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:22.800244 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 469/927] 10.222.0.30 () {56 vars in 1103 bytes} [Wed Nov 2 07:18:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:22.807450 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89360b33-5c55-4278-96df-57d89d3d6958 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:22.920425 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89360b33-5c55-4278-96df-57d89d3d6958 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:22.954420 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f6e34-ec92-4756-8079-30c3ea3b07c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:22.955117 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f6e34-ec92-4756-8079-30c3ea3b07c0 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:22.955371 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f6e34-ec92-4756-8079-30c3ea3b07c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:22.955627 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f6e34-ec92-4756-8079-30c3ea3b07c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:22.967500 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0f6e34-ec92-4756-8079-30c3ea3b07c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:22.969010 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0f6e34-ec92-4756-8079-30c3ea3b07c0 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:22.983663 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-fe0f6e34-ec92-4756-8079-30c3ea3b07c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:22.983663 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:22.983663 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:22.983663 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:22.983663 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:22.983663 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:22.983663 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:22.983663 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:22.983663 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:22.983663 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:22.983663 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:22.983663 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:22.983663 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:18:22.983663 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:18:22.983663 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:18:22.983663 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:18:22.984831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:22.984831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:22.984831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:22.984831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:22.984831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:22.984831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:22.984831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:22.984831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:22.984831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:22.984831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:22.984831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:22.984831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:22.984831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:22.984831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:22.984831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:22.984831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:22.984831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:18:22.985716 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:18:22.985716 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:22.985716 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:18:22.985716 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 459/928] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:22.991179 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e28e3f-8c71-4aaa-a1ae-a75f90ba9eea None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:23.097830 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e28e3f-8c71-4aaa-a1ae-a75f90ba9eea None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:23.116902 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:23.136837 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053407a-0328-4153-8145-d9bcf4855491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:23.137452 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053407a-0328-4153-8145-d9bcf4855491 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:23.137735 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053407a-0328-4153-8145-d9bcf4855491 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:23.137998 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053407a-0328-4153-8145-d9bcf4855491 None admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:23.138633 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5053407a-0328-4153-8145-d9bcf4855491 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:23.138961 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5053407a-0328-4153-8145-d9bcf4855491 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:23.140337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5053407a-0328-4153-8145-d9bcf4855491 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:23.152861 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 470/929] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:18:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:23.158704 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f6e34-ec92-4756-8079-30c3ea3b07c0 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:23.214165 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:23.274092 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f6e34-ec92-4756-8079-30c3ea3b07c0 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:23.315510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab085b7-1266-4772-bccf-fbee8ea180b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:23.316237 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab085b7-1266-4772-bccf-fbee8ea180b1 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:23.316609 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab085b7-1266-4772-bccf-fbee8ea180b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:23.316910 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab085b7-1266-4772-bccf-fbee8ea180b1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:23.326372 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab085b7-1266-4772-bccf-fbee8ea180b1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:23.328393 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab085b7-1266-4772-bccf-fbee8ea180b1 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:23.342401 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-1ab085b7-1266-4772-bccf-fbee8ea180b1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Nov 02 07:18:23.342401 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:23.342401 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:23.342401 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:23.342401 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:23.342401 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:23.342401 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:23.342401 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:23.342401 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:23.342401 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:23.342401 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:23.342401 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:23.342401 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:18:23.342401 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:18:23.342401 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:18:23.342401 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:18:23.343559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:23.343559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:23.343559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:18:23.343559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:18:23.343559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:18:23.343559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:18:23.343559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:23.343559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:23.343559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:23.343559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:23.343559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:23.343559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:23.343559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:23.343559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:23.343559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:23.343559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:23.343559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:23.343559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:23.344718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:23.344718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:23.344718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:18:23.344718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:18:23.344718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:18:23.344718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:18:23.344718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:18:23.344718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:18:23.344718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Nov 02 07:18:23.344718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:18:23.344718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 460/930] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:23.352838 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053407a-0328-4153-8145-d9bcf4855491 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:23.463543 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053407a-0328-4153-8145-d9bcf4855491 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:23.498384 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052c767-311f-4a4a-a31d-27f457406155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:23.499130 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052c767-311f-4a4a-a31d-27f457406155 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:23.499541 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052c767-311f-4a4a-a31d-27f457406155 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:23.499916 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052c767-311f-4a4a-a31d-27f457406155 None admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:23.500636 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052c767-311f-4a4a-a31d-27f457406155 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:23.500939 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052c767-311f-4a4a-a31d-27f457406155 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:23.502492 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052c767-311f-4a4a-a31d-27f457406155 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:23.524261 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 471/931] 10.222.0.30 () {56 vars in 1143 bytes} [Wed Nov 2 07:18:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:23.530992 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab085b7-1266-4772-bccf-fbee8ea180b1 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:23.647996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab085b7-1266-4772-bccf-fbee8ea180b1 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:23.658968 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:23.684057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f055672-1d14-4e9f-91fc-d50fe3a5ad09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:23.684665 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f055672-1d14-4e9f-91fc-d50fe3a5ad09 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:23.684941 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f055672-1d14-4e9f-91fc-d50fe3a5ad09 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:23.685209 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f055672-1d14-4e9f-91fc-d50fe3a5ad09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:23.755640 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f055672-1d14-4e9f-91fc-d50fe3a5ad09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:23.757089 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f055672-1d14-4e9f-91fc-d50fe3a5ad09 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:23.865533 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 461/932] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 07:18:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:18:23.871434 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052c767-311f-4a4a-a31d-27f457406155 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:23.907832 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:23.977982 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052c767-311f-4a4a-a31d-27f457406155 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:24.013052 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52261e-01f5-46d0-8f74-9a44ed669bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:24.013687 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52261e-01f5-46d0-8f74-9a44ed669bbc None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:24.013966 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52261e-01f5-46d0-8f74-9a44ed669bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:24.014240 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52261e-01f5-46d0-8f74-9a44ed669bbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:24.024784 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e52261e-01f5-46d0-8f74-9a44ed669bbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:24.026242 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e52261e-01f5-46d0-8f74-9a44ed669bbc None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:24.041499 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-8e52261e-01f5-46d0-8f74-9a44ed669bbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:24.041499 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:24.041499 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:24.041499 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:24.041499 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:24.041499 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:24.041499 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:24.041499 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:24.041499 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:24.041499 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:24.041499 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:24.041499 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:24.041499 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:18:24.041499 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:18:24.041499 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:18:24.041499 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:18:24.042592 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:24.042592 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:24.042592 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:24.042592 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:24.042592 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:24.042592 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:24.042592 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:24.042592 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:24.042592 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:24.042592 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:24.042592 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:24.042592 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:24.042592 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:24.042592 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:24.042592 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:24.042592 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:24.042592 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:18:24.043519 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:18:24.043519 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:24.043519 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:18:24.043519 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 472/933] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:24.048595 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f055672-1d14-4e9f-91fc-d50fe3a5ad09 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:24.060385 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:24.154338 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f055672-1d14-4e9f-91fc-d50fe3a5ad09 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:24.188770 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde4d3f-2406-40aa-8af1-c6d6776d4615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:24.189657 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde4d3f-2406-40aa-8af1-c6d6776d4615 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:24.190031 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde4d3f-2406-40aa-8af1-c6d6776d4615 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:24.190419 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde4d3f-2406-40aa-8af1-c6d6776d4615 None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:24.191096 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbde4d3f-2406-40aa-8af1-c6d6776d4615 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:24.191468 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbde4d3f-2406-40aa-8af1-c6d6776d4615 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:24.193032 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbde4d3f-2406-40aa-8af1-c6d6776d4615 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:24.205666 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 462/934] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:18:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:24.211477 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52261e-01f5-46d0-8f74-9a44ed669bbc None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:24.305423 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:24.321052 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52261e-01f5-46d0-8f74-9a44ed669bbc None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:24.355611 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b20aa-1b77-45ad-8989-5f2575c948d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:24.356258 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b20aa-1b77-45ad-8989-5f2575c948d6 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:24.356513 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b20aa-1b77-45ad-8989-5f2575c948d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:24.356826 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b20aa-1b77-45ad-8989-5f2575c948d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:24.364030 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b20aa-1b77-45ad-8989-5f2575c948d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:24.365464 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b20aa-1b77-45ad-8989-5f2575c948d6 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:24.377470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-750b20aa-1b77-45ad-8989-5f2575c948d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:18:24.377470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:24.377470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:24.377470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:24.377470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:24.377470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:24.377470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:24.377470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:24.377470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:24.377470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:24.377470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:24.377470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:24.377470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:18:24.377470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:18:24.377470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:18:24.378405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:18:24.378405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:24.378405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:24.378405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:24.378405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:24.378405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:24.378405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:24.378405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:24.378405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:24.378405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:24.378405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:24.378405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:24.378405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:24.378405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:24.378405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:24.378405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:24.378405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:24.379334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:18:24.379334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:18:24.379334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:18:24.379334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:18:24.379334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:18:24.379334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:18:24.379334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:18:24.379334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:18:24.379334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 473/935] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:18:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:24.384987 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde4d3f-2406-40aa-8af1-c6d6776d4615 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:24.453765 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:24.487088 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde4d3f-2406-40aa-8af1-c6d6776d4615 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:24.525275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbcb99-3c5f-47d3-ad2d-74bf779c3f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:24.526109 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbcb99-3c5f-47d3-ad2d-74bf779c3f40 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:24.526433 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbcb99-3c5f-47d3-ad2d-74bf779c3f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:24.526981 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbcb99-3c5f-47d3-ad2d-74bf779c3f40 None admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:24.527973 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafbcb99-3c5f-47d3-ad2d-74bf779c3f40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:24.528561 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafbcb99-3c5f-47d3-ad2d-74bf779c3f40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:24.530969 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafbcb99-3c5f-47d3-ad2d-74bf779c3f40 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:24.546752 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 463/936] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 07:18:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:24.553176 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b20aa-1b77-45ad-8989-5f2575c948d6 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:24.662092 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b20aa-1b77-45ad-8989-5f2575c948d6 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:24.696370 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900b087b-f533-46b2-8ce5-4b52267f08d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:24.696902 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b087b-f533-46b2-8ce5-4b52267f08d6 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:24.697197 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b087b-f533-46b2-8ce5-4b52267f08d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:24.697493 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b087b-f533-46b2-8ce5-4b52267f08d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:24.698177 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900b087b-f533-46b2-8ce5-4b52267f08d6 None admin] RBAC: Adding query filter params (role.id=268fecadd98446f2b7d41755f3fe3649, scope.project.id=25e2bc8417754f909b416b93d5b7afb7, user.id=932d3076fb4b426d8c83d02eb60d8063) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:24.698474 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900b087b-f533-46b2-8ce5-4b52267f08d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:24.699976 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900b087b-f533-46b2-8ce5-4b52267f08d6 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:24.706801 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 474/937] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 07:18:24 2022] GET /identity/v3/role_assignments?user.id=932d3076fb4b426d8c83d02eb60d8063&scope.project.id=25e2bc8417754f909b416b93d5b7afb7&role.id=268fecadd98446f2b7d41755f3fe3649 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:26.899205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0ab40-874e-4dc3-b555-489daa7053b2 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:26.899760 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0ab40-874e-4dc3-b555-489daa7053b2 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:26.899760 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0ab40-874e-4dc3-b555-489daa7053b2 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:26.901025 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 464/938] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:26.906998 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a3b37-2a8c-45c9-abde-dfad9f8f4ec2 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:26.907455 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a3b37-2a8c-45c9-abde-dfad9f8f4ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:26.907580 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a3b37-2a8c-45c9-abde-dfad9f8f4ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:26.928359 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-7c9a3b37-2a8c-45c9-abde-dfad9f8f4ec2 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:26.929585 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-7c9a3b37-2a8c-45c9-abde-dfad9f8f4ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:26.932898 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:27.004721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 475/939] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:18:27.273367 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0ab40-874e-4dc3-b555-489daa7053b2 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:27.393352 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0ab40-874e-4dc3-b555-489daa7053b2 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:27.427136 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:27.433480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b9bb9-d454-40fc-8836-395bb2dc5cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:27.434073 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b9bb9-d454-40fc-8836-395bb2dc5cff None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:27.434344 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b9bb9-d454-40fc-8836-395bb2dc5cff None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:27.434608 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b9bb9-d454-40fc-8836-395bb2dc5cff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:27.445580 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101b9bb9-d454-40fc-8836-395bb2dc5cff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:18:27.450291 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-101b9bb9-d454-40fc-8836-395bb2dc5cff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:18:27.450291 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:27.450291 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:27.450291 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:27.450291 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:27.450291 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:27.450291 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:27.450291 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:27.450291 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:27.450291 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:27.450291 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:27.450291 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:27.450291 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:18:27.450291 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:18:27.450291 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:18:27.450291 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:18:27.451676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:18:27.451676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:18:27.451676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:18:27.451676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:18:27.451676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:27.451676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:27.451676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:27.451676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:27.451676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:27.451676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:27.451676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:27.451676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:27.451676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:27.451676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:27.451676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:27.451676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:27.452802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:27.452802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:27.452802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:27.452802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:27.452802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:18:27.452802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:18:27.452802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:18:27.452802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:18:27.452802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:18:27.452802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:18:27.452802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:18:27.452802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:18:27.456627 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 465/940] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 07:18:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:18:27.457714 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a3b37-2a8c-45c9-abde-dfad9f8f4ec2 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:27.566266 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a3b37-2a8c-45c9-abde-dfad9f8f4ec2 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:27.608065 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3929241-1d05-4438-89af-65166925c9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:27.608751 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3929241-1d05-4438-89af-65166925c9af None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:27.609049 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3929241-1d05-4438-89af-65166925c9af None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:27.609320 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3929241-1d05-4438-89af-65166925c9af None admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:27.610043 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3929241-1d05-4438-89af-65166925c9af None admin] RBAC: Adding query filter params (name=admin) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:27.610337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3929241-1d05-4438-89af-65166925c9af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:27.612600 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3929241-1d05-4438-89af-65166925c9af None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:27.621603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 476/941] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:18:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:27.628482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b9bb9-d454-40fc-8836-395bb2dc5cff None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:27.737757 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b9bb9-d454-40fc-8836-395bb2dc5cff None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:27.773366 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93241325-be7e-4677-bff5-1ec9b4eace71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:27.774001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93241325-be7e-4677-bff5-1ec9b4eace71 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:27.774277 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93241325-be7e-4677-bff5-1ec9b4eace71 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:27.774579 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93241325-be7e-4677-bff5-1ec9b4eace71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:27.784989 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93241325-be7e-4677-bff5-1ec9b4eace71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:27.786444 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93241325-be7e-4677-bff5-1ec9b4eace71 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:27.801029 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-93241325-be7e-4677-bff5-1ec9b4eace71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:27.801029 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:27.801029 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:27.801029 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:27.801029 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:27.801029 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:27.801029 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:27.801029 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:27.801029 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:27.801029 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:27.801029 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:27.801029 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:27.801029 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:18:27.801029 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:18:27.801029 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:18:27.801029 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:18:27.802108 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:27.802108 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:27.802108 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:27.802108 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:27.802108 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:27.802108 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:27.802108 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:27.802108 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:27.802108 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:27.802108 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:27.802108 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:27.802108 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:27.802108 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:27.802108 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:27.802108 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:27.802108 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:27.802108 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:18:27.803027 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:18:27.803027 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:27.803027 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:18:27.803027 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 466/942] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:18:27.809256 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3929241-1d05-4438-89af-65166925c9af None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:27.812392 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:27.915852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3929241-1d05-4438-89af-65166925c9af None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:27.950103 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b146b5b-f576-43cc-b903-86eab8037d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:27.950710 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b146b5b-f576-43cc-b903-86eab8037d0f None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:27.950991 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b146b5b-f576-43cc-b903-86eab8037d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:27.951263 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b146b5b-f576-43cc-b903-86eab8037d0f None admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:27.951903 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b146b5b-f576-43cc-b903-86eab8037d0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:27.952236 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b146b5b-f576-43cc-b903-86eab8037d0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:27.953571 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b146b5b-f576-43cc-b903-86eab8037d0f None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:27.966115 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 477/943] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:18:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:27.972900 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93241325-be7e-4677-bff5-1ec9b4eace71 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:28.020910 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:28.080463 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93241325-be7e-4677-bff5-1ec9b4eace71 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:28.115845 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec524644-e4fb-418e-a7a1-0798eaff779f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:28.116576 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec524644-e4fb-418e-a7a1-0798eaff779f None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:28.116875 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec524644-e4fb-418e-a7a1-0798eaff779f None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:28.117115 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec524644-e4fb-418e-a7a1-0798eaff779f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:28.125243 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec524644-e4fb-418e-a7a1-0798eaff779f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:28.126794 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec524644-e4fb-418e-a7a1-0798eaff779f None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:28.138535 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-ec524644-e4fb-418e-a7a1-0798eaff779f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Nov 02 07:18:28.138535 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:28.138535 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:28.138535 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:28.138535 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:28.138535 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:28.138535 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:28.138535 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:28.138535 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:28.138535 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:28.138535 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:28.138535 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:28.138535 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:18:28.138535 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:18:28.138535 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:18:28.138535 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:18:28.139743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:28.139743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:28.139743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:18:28.139743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:18:28.139743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:18:28.139743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:18:28.139743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:28.139743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:28.139743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:28.139743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:28.139743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:28.139743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:28.139743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:28.139743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:28.139743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:28.139743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:28.139743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:28.139743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:28.140889 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:28.140889 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:28.140889 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:18:28.140889 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:18:28.140889 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:18:28.140889 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:18:28.140889 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:18:28.140889 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:18:28.140889 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Nov 02 07:18:28.140889 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:18:28.142252 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 467/944] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:18:28.149010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b146b5b-f576-43cc-b903-86eab8037d0f None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:28.211915 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:28.253946 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b146b5b-f576-43cc-b903-86eab8037d0f None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:28.295213 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea4c725-24ab-4345-9143-685616ec390b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:28.295832 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea4c725-24ab-4345-9143-685616ec390b None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:28.296102 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea4c725-24ab-4345-9143-685616ec390b None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:28.296405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea4c725-24ab-4345-9143-685616ec390b None admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:28.297082 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea4c725-24ab-4345-9143-685616ec390b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:28.297380 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea4c725-24ab-4345-9143-685616ec390b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:28.298934 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea4c725-24ab-4345-9143-685616ec390b None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:28.317525 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 478/945] 10.222.0.30 () {56 vars in 1143 bytes} [Wed Nov 2 07:18:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:28.323533 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec524644-e4fb-418e-a7a1-0798eaff779f None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:28.443683 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec524644-e4fb-418e-a7a1-0798eaff779f None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:28.445657 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:28.481984 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe95fa-967e-4d59-80c8-9ee9ec43c766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:28.482517 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe95fa-967e-4d59-80c8-9ee9ec43c766 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:28.482824 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe95fa-967e-4d59-80c8-9ee9ec43c766 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:28.483093 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe95fa-967e-4d59-80c8-9ee9ec43c766 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:28.494671 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fe95fa-967e-4d59-80c8-9ee9ec43c766 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:28.496125 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fe95fa-967e-4d59-80c8-9ee9ec43c766 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:28.509907 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-c8fe95fa-967e-4d59-80c8-9ee9ec43c766 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:28.509907 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:28.509907 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:28.509907 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:28.509907 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:28.509907 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:28.509907 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:28.509907 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:28.509907 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:28.509907 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:28.509907 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:28.509907 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:28.509907 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:18:28.509907 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:18:28.509907 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:18:28.509907 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:18:28.511121 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:28.511121 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:28.511121 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:28.511121 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:28.511121 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:28.511121 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:28.511121 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:28.511121 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:28.511121 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:28.511121 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:28.511121 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:28.511121 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:28.511121 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:28.511121 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:28.511121 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:28.511121 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:28.511121 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:18:28.512163 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:18:28.512163 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:28.512163 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:18:28.512163 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 468/946] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:28.517163 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea4c725-24ab-4345-9143-685616ec390b None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:28.626836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea4c725-24ab-4345-9143-685616ec390b None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:28.651085 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:28.666180 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8d01a-33c6-441d-bef1-ee29483abc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:28.666955 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8d01a-33c6-441d-bef1-ee29483abc82 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:28.667245 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8d01a-33c6-441d-bef1-ee29483abc82 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:28.667556 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8d01a-33c6-441d-bef1-ee29483abc82 None admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:28.668257 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e8d01a-33c6-441d-bef1-ee29483abc82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:28.668637 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e8d01a-33c6-441d-bef1-ee29483abc82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:28.670207 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e8d01a-33c6-441d-bef1-ee29483abc82 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:28.683313 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 479/947] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:18:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:18:28.689704 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe95fa-967e-4d59-80c8-9ee9ec43c766 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:28.790826 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe95fa-967e-4d59-80c8-9ee9ec43c766 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:28.829575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f4388-820e-49e9-bac4-b1402831e4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:28.830435 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f4388-820e-49e9-bac4-b1402831e4e5 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:28.830749 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f4388-820e-49e9-bac4-b1402831e4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:28.831024 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f4388-820e-49e9-bac4-b1402831e4e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:28.838235 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f4388-820e-49e9-bac4-b1402831e4e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:28.839754 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f4388-820e-49e9-bac4-b1402831e4e5 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:28.852157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-3a6f4388-820e-49e9-bac4-b1402831e4e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:18:28.852157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:28.852157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:28.852157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:28.852157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:28.852157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:28.852157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:28.852157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:28.852157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:28.852157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:28.852157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:28.852157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:28.852157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:18:28.852157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:18:28.852157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:18:28.853166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:18:28.853166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:28.853166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:28.853166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:28.853166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:28.853166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:28.853166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:28.853166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:28.853166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:28.853166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:28.853166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:28.853166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:28.853166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:28.853166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:28.853166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:28.853166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:28.853166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:28.854045 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:18:28.854045 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:18:28.854045 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:18:28.854045 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:18:28.854045 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:18:28.854045 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:18:28.854045 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:18:28.854045 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:18:28.854045 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 469/948] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:18:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:28.859894 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8d01a-33c6-441d-bef1-ee29483abc82 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:28.975201 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8d01a-33c6-441d-bef1-ee29483abc82 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:29.010507 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6015f4ba-6c8a-44f5-8c16-7af36d50d766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:29.011077 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015f4ba-6c8a-44f5-8c16-7af36d50d766 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:29.011352 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015f4ba-6c8a-44f5-8c16-7af36d50d766 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:29.011619 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015f4ba-6c8a-44f5-8c16-7af36d50d766 None admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:29.012305 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015f4ba-6c8a-44f5-8c16-7af36d50d766 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:29.012609 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015f4ba-6c8a-44f5-8c16-7af36d50d766 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:29.014083 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015f4ba-6c8a-44f5-8c16-7af36d50d766 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:29.028648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 480/949] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 07:18:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:29.034828 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f4388-820e-49e9-bac4-b1402831e4e5 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:29.052708 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:29.140351 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f4388-820e-49e9-bac4-b1402831e4e5 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:29.177895 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8d45c-f0e4-45ca-a0fa-b9e151434d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:29.178582 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8d45c-f0e4-45ca-a0fa-b9e151434d75 None admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:29.178901 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8d45c-f0e4-45ca-a0fa-b9e151434d75 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:29.179171 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8d45c-f0e4-45ca-a0fa-b9e151434d75 None admin] PATH_INFO: `/v3/projects/25e2bc8417754f909b416b93d5b7afb7/users/932d3076fb4b426d8c83d02eb60d8063/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:29.181248 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b8d45c-f0e4-45ca-a0fa-b9e151434d75 None admin] RBAC: Authorizing `identity:create_grant(project_id=25e2bc8417754f909b416b93d5b7afb7, user_id=932d3076fb4b426d8c83d02eb60d8063, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:29.182971 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b8d45c-f0e4-45ca-a0fa-b9e151434d75 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:29.197747 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 470/950] 10.222.0.30 () {58 vars in 1439 bytes} [Wed Nov 2 07:18:29 2022] PUT /identity/v3/projects/25e2bc8417754f909b416b93d5b7afb7/users/932d3076fb4b426d8c83d02eb60d8063/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:18:31.512177 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814017b7-4d4d-4818-9751-f7a67e777a48 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:31.512757 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814017b7-4d4d-4818-9751-f7a67e777a48 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:31.512757 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814017b7-4d4d-4818-9751-f7a67e777a48 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:31.514081 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 481/951] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:31.520453 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a08b4-fc22-4039-a9a7-b3451673ae20 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:31.521019 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a08b4-fc22-4039-a9a7-b3451673ae20 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:31.521019 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a08b4-fc22-4039-a9a7-b3451673ae20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:31.541529 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-e91a08b4-fc22-4039-a9a7-b3451673ae20 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:31.542872 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-e91a08b4-fc22-4039-a9a7-b3451673ae20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:31.618515 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 471/952] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 07:18:31.914164 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814017b7-4d4d-4818-9751-f7a67e777a48 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:31.962054 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:32.025067 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814017b7-4d4d-4818-9751-f7a67e777a48 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:32.060539 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3bd71-67ae-4528-8c49-03d776267eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:32.061152 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3bd71-67ae-4528-8c49-03d776267eec None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:32.061436 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3bd71-67ae-4528-8c49-03d776267eec None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:32.061745 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3bd71-67ae-4528-8c49-03d776267eec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:32.072563 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b3bd71-67ae-4528-8c49-03d776267eec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 07:18:32.077555 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-56b3bd71-67ae-4528-8c49-03d776267eec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:18:32.077555 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:32.077555 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:32.077555 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:32.077555 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:32.077555 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:32.077555 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:32.077555 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:32.077555 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:32.077555 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:32.077555 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:32.077555 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:32.077555 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 07:18:32.077555 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 07:18:32.077555 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 07:18:32.077555 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 07:18:32.078598 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 07:18:32.078598 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 07:18:32.078598 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 07:18:32.078598 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 07:18:32.078598 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:32.078598 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:32.078598 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:32.078598 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:32.078598 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:32.078598 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:32.078598 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:32.078598 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:32.078598 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:32.078598 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:32.078598 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:32.078598 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:32.079629 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:32.079629 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:32.079629 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:32.079629 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:32.079629 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 07:18:32.079629 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 07:18:32.079629 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 07:18:32.079629 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 07:18:32.079629 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 07:18:32.079629 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 07:18:32.079629 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 07:18:32.079629 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:18:32.079629 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 482/953] 10.222.0.30 () {56 vars in 1096 bytes} [Wed Nov 2 07:18:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:32.084999 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a08b4-fc22-4039-a9a7-b3451673ae20 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:32.088144 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:32.191781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a08b4-fc22-4039-a9a7-b3451673ae20 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:32.226467 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf32e0-4078-42e3-95a8-f2e38638714f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:32.227124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf32e0-4078-42e3-95a8-f2e38638714f None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:32.227451 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf32e0-4078-42e3-95a8-f2e38638714f None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:32.227752 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf32e0-4078-42e3-95a8-f2e38638714f None admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:32.228516 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf32e0-4078-42e3-95a8-f2e38638714f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:32.228832 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf32e0-4078-42e3-95a8-f2e38638714f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:32.230272 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf32e0-4078-42e3-95a8-f2e38638714f None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:32.239808 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 472/954] 10.222.0.30 () {56 vars in 1099 bytes} [Wed Nov 2 07:18:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:32.245800 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3bd71-67ae-4528-8c49-03d776267eec None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:32.366129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:32.369727 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3bd71-67ae-4528-8c49-03d776267eec None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:32.405588 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83825898-d387-4e37-a839-1215284e9044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:32.406220 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83825898-d387-4e37-a839-1215284e9044 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:32.406485 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83825898-d387-4e37-a839-1215284e9044 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:32.406845 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83825898-d387-4e37-a839-1215284e9044 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:32.417256 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83825898-d387-4e37-a839-1215284e9044 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:32.418747 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83825898-d387-4e37-a839-1215284e9044 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:32.433147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-83825898-d387-4e37-a839-1215284e9044 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:32.433147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:32.433147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:32.433147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:32.433147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:32.433147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:32.433147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:32.433147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:32.433147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:32.433147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:32.433147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:32.433147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:32.433147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:18:32.433147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:18:32.433147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:18:32.433147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:18:32.434385 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:32.434385 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:32.434385 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:32.434385 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:32.434385 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:32.434385 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:32.434385 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:32.434385 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:32.434385 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:32.434385 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:32.434385 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:32.434385 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:32.434385 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:32.434385 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:32.434385 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:32.434385 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:32.434385 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:18:32.436081 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:18:32.436081 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:32.436081 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:18:32.436081 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 483/955] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:32.440301 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf32e0-4078-42e3-95a8-f2e38638714f None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:32.512826 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:32.551259 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf32e0-4078-42e3-95a8-f2e38638714f None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:32.585028 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604b938-8f86-4bf5-a184-f61a5ecb31b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:32.585638 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604b938-8f86-4bf5-a184-f61a5ecb31b4 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:32.585907 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604b938-8f86-4bf5-a184-f61a5ecb31b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:32.586170 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604b938-8f86-4bf5-a184-f61a5ecb31b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:32.586827 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8604b938-8f86-4bf5-a184-f61a5ecb31b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:32.587167 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8604b938-8f86-4bf5-a184-f61a5ecb31b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:32.588493 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8604b938-8f86-4bf5-a184-f61a5ecb31b4 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:32.602229 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 473/956] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:32.607788 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83825898-d387-4e37-a839-1215284e9044 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:32.716073 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83825898-d387-4e37-a839-1215284e9044 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:32.750733 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc93177-0599-4b28-912a-303fc31c45fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:32.751364 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc93177-0599-4b28-912a-303fc31c45fb None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:32.751644 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc93177-0599-4b28-912a-303fc31c45fb None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:32.751909 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc93177-0599-4b28-912a-303fc31c45fb None admin] PATH_INFO: `/v3/users/placement` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:32.759959 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc93177-0599-4b28-912a-303fc31c45fb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:32.761407 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc93177-0599-4b28-912a-303fc31c45fb None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:32.773161 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-2bc93177-0599-4b28-912a-303fc31c45fb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Nov 02 07:18:32.773161 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:32.773161 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:32.773161 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:32.773161 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:32.773161 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:32.773161 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:32.773161 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:32.773161 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:32.773161 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:32.773161 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:32.773161 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:32.773161 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 07:18:32.773161 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 07:18:32.773161 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 07:18:32.773161 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 07:18:32.779250 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:32.779250 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:32.779250 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 07:18:32.779250 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:18:32.779250 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 07:18:32.779250 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 07:18:32.779250 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:32.779250 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:32.779250 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:32.779250 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:32.779250 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:32.779250 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:32.779250 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:32.779250 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:32.779250 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:32.779250 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:32.779250 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:32.779250 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:32.780332 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:32.780332 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:32.780332 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 07:18:32.780332 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 07:18:32.780332 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 07:18:32.780332 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 07:18:32.780332 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 07:18:32.780332 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 07:18:32.780332 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Nov 02 07:18:32.780332 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:18:32.780332 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 484/957] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 07:18:32.780935 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604b938-8f86-4bf5-a184-f61a5ecb31b4 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:32.885843 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604b938-8f86-4bf5-a184-f61a5ecb31b4 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:32.914289 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:32.923691 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a29a3f-6b1c-477a-adaa-5f21a5e2f0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:32.926399 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a29a3f-6b1c-477a-adaa-5f21a5e2f0a2 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:32.926734 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a29a3f-6b1c-477a-adaa-5f21a5e2f0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:32.927068 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a29a3f-6b1c-477a-adaa-5f21a5e2f0a2 None admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:32.927857 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a29a3f-6b1c-477a-adaa-5f21a5e2f0a2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:32.928218 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a29a3f-6b1c-477a-adaa-5f21a5e2f0a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:32.930067 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a29a3f-6b1c-477a-adaa-5f21a5e2f0a2 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:32.956112 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 474/958] 10.222.0.30 () {56 vars in 1143 bytes} [Wed Nov 2 07:18:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:32.957784 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc93177-0599-4b28-912a-303fc31c45fb None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:32.974443 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:33.085623 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc93177-0599-4b28-912a-303fc31c45fb None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:33.127382 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8516339-0a0a-4dab-9eed-6592620f1536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:33.128074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8516339-0a0a-4dab-9eed-6592620f1536 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:33.128385 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8516339-0a0a-4dab-9eed-6592620f1536 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:33.128652 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8516339-0a0a-4dab-9eed-6592620f1536 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:33.203984 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8516339-0a0a-4dab-9eed-6592620f1536 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:33.205862 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8516339-0a0a-4dab-9eed-6592620f1536 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:33.221400 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:33.321907 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 485/959] 10.222.0.30 () {58 vars in 1282 bytes} [Wed Nov 2 07:18:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 07:18:33.327986 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a29a3f-6b1c-477a-adaa-5f21a5e2f0a2 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:33.434426 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a29a3f-6b1c-477a-adaa-5f21a5e2f0a2 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:33.472690 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6da5cc-b5eb-4f22-b3aa-e335972634be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:33.473465 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6da5cc-b5eb-4f22-b3aa-e335972634be None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:33.473738 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6da5cc-b5eb-4f22-b3aa-e335972634be None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:33.473999 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6da5cc-b5eb-4f22-b3aa-e335972634be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:33.485686 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6da5cc-b5eb-4f22-b3aa-e335972634be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:33.487579 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6da5cc-b5eb-4f22-b3aa-e335972634be None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:33.514093 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-6c6da5cc-b5eb-4f22-b3aa-e335972634be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:33.514093 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:33.514093 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:33.514093 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:33.514093 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:33.514093 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:33.514093 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:33.514093 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:33.514093 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:33.514093 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:33.514093 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:33.514093 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:33.514093 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 07:18:33.514093 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 07:18:33.514093 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 07:18:33.514093 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 07:18:33.520963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:33.520963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:33.520963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:33.520963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:33.520963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:33.520963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:33.520963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:33.520963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:33.520963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:33.520963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:33.520963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:33.520963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:33.520963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:33.520963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:33.520963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:33.520963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:33.520963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 07:18:33.522104 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 07:18:33.522104 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 07:18:33.522104 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:18:33.522104 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 475/960] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:18:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:33.522392 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8516339-0a0a-4dab-9eed-6592620f1536 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:33.630542 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8516339-0a0a-4dab-9eed-6592620f1536 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:33.646118 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:33.671598 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227fea83-702c-454b-a35f-baf8b41baf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:33.672240 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fea83-702c-454b-a35f-baf8b41baf34 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:33.672516 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fea83-702c-454b-a35f-baf8b41baf34 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:33.672769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fea83-702c-454b-a35f-baf8b41baf34 None admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:33.673422 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227fea83-702c-454b-a35f-baf8b41baf34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:33.673720 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227fea83-702c-454b-a35f-baf8b41baf34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:33.675266 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227fea83-702c-454b-a35f-baf8b41baf34 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:33.688230 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 486/961] 10.222.0.30 () {56 vars in 1109 bytes} [Wed Nov 2 07:18:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:18:33.694340 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6da5cc-b5eb-4f22-b3aa-e335972634be None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:33.734359 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:33.805875 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6da5cc-b5eb-4f22-b3aa-e335972634be None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:33.841583 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb4db7-d20f-4339-89c0-6bf04704c675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:33.842207 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb4db7-d20f-4339-89c0-6bf04704c675 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:33.842483 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb4db7-d20f-4339-89c0-6bf04704c675 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:33.842786 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb4db7-d20f-4339-89c0-6bf04704c675 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:33.850857 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb4db7-d20f-4339-89c0-6bf04704c675 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:33.852336 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb4db7-d20f-4339-89c0-6bf04704c675 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:33.864753 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application [None req-c9bb4db7-d20f-4339-89c0-6bf04704c675 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:18:33.864753 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:33.864753 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:33.864753 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:33.864753 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:33.864753 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:33.864753 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:33.864753 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:33.864753 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:33.864753 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:33.864753 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:33.864753 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:33.864753 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 07:18:33.864753 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 07:18:33.864753 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 07:18:33.865748 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 07:18:33.865748 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:33.865748 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:33.865748 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:33.865748 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:33.865748 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:33.865748 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:33.865748 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:33.865748 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:33.865748 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:33.865748 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:33.865748 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:33.865748 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:33.865748 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:33.865748 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:33.865748 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:33.865748 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:33.866650 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 07:18:33.866650 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 07:18:33.866650 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 07:18:33.866650 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 07:18:33.866650 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 07:18:33.866650 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 07:18:33.866650 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 07:18:33.866650 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: ERROR keystone.server.flask.application Nov 02 07:18:33.866650 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 476/962] 10.222.0.30 () {56 vars in 1111 bytes} [Wed Nov 2 07:18:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:33.872230 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227fea83-702c-454b-a35f-baf8b41baf34 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:33.980296 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227fea83-702c-454b-a35f-baf8b41baf34 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:34.014113 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32da646-9b27-4e5f-8114-09e421b491e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:34.014773 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32da646-9b27-4e5f-8114-09e421b491e7 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:34.015046 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32da646-9b27-4e5f-8114-09e421b491e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:34.015313 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32da646-9b27-4e5f-8114-09e421b491e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:34.015985 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32da646-9b27-4e5f-8114-09e421b491e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:34.016287 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32da646-9b27-4e5f-8114-09e421b491e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:34.017770 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32da646-9b27-4e5f-8114-09e421b491e7 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:34.030523 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 487/963] 10.222.0.30 () {56 vars in 1148 bytes} [Wed Nov 2 07:18:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:34.037096 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb4db7-d20f-4339-89c0-6bf04704c675 None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:34.133973 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:34.140440 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb4db7-d20f-4339-89c0-6bf04704c675 None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:34.174157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de05b51-e80d-4357-9c02-efaca8253d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:34.174795 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de05b51-e80d-4357-9c02-efaca8253d74 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:34.175074 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de05b51-e80d-4357-9c02-efaca8253d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:34.175341 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de05b51-e80d-4357-9c02-efaca8253d74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:34.176083 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de05b51-e80d-4357-9c02-efaca8253d74 None admin] RBAC: Adding query filter params (role.id=1e35bb725eb8475ea63e2d9538ab67d5, scope.project.id=25e2bc8417754f909b416b93d5b7afb7, user.id=932d3076fb4b426d8c83d02eb60d8063) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:34.176388 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de05b51-e80d-4357-9c02-efaca8253d74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:34.177832 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de05b51-e80d-4357-9c02-efaca8253d74 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:34.183660 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 477/964] 10.222.0.30 () {56 vars in 1374 bytes} [Wed Nov 2 07:18:34 2022] GET /identity/v3/role_assignments?user.id=932d3076fb4b426d8c83d02eb60d8063&scope.project.id=25e2bc8417754f909b416b93d5b7afb7&role.id=1e35bb725eb8475ea63e2d9538ab67d5 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:36.354701 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b67dd-84ba-46f6-b387-c57f0efa1101 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:36.355269 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b67dd-84ba-46f6-b387-c57f0efa1101 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:36.355269 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b67dd-84ba-46f6-b387-c57f0efa1101 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:36.356646 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 488/965] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:36.362064 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c80775-b329-4130-b195-3a57e51539e3 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:36.362515 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c80775-b329-4130-b195-3a57e51539e3 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:36.362620 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c80775-b329-4130-b195-3a57e51539e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:36.384310 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-51c80775-b329-4130-b195-3a57e51539e3 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:36.385505 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-51c80775-b329-4130-b195-3a57e51539e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:36.457485 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 478/966] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:18:36.721134 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b67dd-84ba-46f6-b387-c57f0efa1101 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:36.760827 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:36.840384 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b67dd-84ba-46f6-b387-c57f0efa1101 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:36.878360 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520175a3-96dd-4e04-895a-cd6e0ee80f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:36.878949 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520175a3-96dd-4e04-895a-cd6e0ee80f7f None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:36.879227 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520175a3-96dd-4e04-895a-cd6e0ee80f7f None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:36.879498 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520175a3-96dd-4e04-895a-cd6e0ee80f7f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:36.880070 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520175a3-96dd-4e04-895a-cd6e0ee80f7f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:36.881451 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520175a3-96dd-4e04-895a-cd6e0ee80f7f None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:36.890687 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-520175a3-96dd-4e04-895a-cd6e0ee80f7f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:18:36.890687 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:36.890687 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:18:36.890687 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:18:36.890687 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:18:36.890687 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:18:36.890687 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:18:36.890687 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:18:36.890687 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:18:36.890687 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:18:36.890687 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:18:36.890687 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:18:36.890687 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:36.890687 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:36.890687 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:36.890687 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:36.890687 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:36.891805 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:36.891805 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:36.891805 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:36.891805 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:36.891805 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:36.891805 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:36.891805 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:18:36.891805 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:18:36.891805 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:18:36.891805 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:18:36.891805 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:36.891805 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:36.891805 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:36.891805 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:36.891805 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:36.891805 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:36.892821 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:36.892821 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:36.892821 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:36.892821 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:36.892821 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:36.892821 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:36.892821 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:36.892821 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:36.892821 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:36.892821 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:36.892821 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:18:36.892821 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:18:36.892821 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:18:36.892821 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:18:36.892821 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 489/967] 10.222.0.30 () {56 vars in 1117 bytes} [Wed Nov 2 07:18:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:36.898712 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c80775-b329-4130-b195-3a57e51539e3 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:36.994230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:37.005665 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c80775-b329-4130-b195-3a57e51539e3 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:37.046786 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b749017-4764-4622-ac76-33852b599033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:37.047418 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b749017-4764-4622-ac76-33852b599033 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:37.047675 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b749017-4764-4622-ac76-33852b599033 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:37.047958 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b749017-4764-4622-ac76-33852b599033 None admin] PATH_INFO: `/v3/services` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:37.048577 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b749017-4764-4622-ac76-33852b599033 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:37.048866 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b749017-4764-4622-ac76-33852b599033 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:37.050802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b749017-4764-4622-ac76-33852b599033 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:37.056427 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 479/968] 10.222.0.30 () {56 vars in 1116 bytes} [Wed Nov 2 07:18:36 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:37.065049 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520175a3-96dd-4e04-895a-cd6e0ee80f7f None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:37.201530 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:37.208724 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520175a3-96dd-4e04-895a-cd6e0ee80f7f None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:37.249312 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085062d8-c0f5-41b2-986c-ca5f4a99c27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:37.250035 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085062d8-c0f5-41b2-986c-ca5f4a99c27c None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:37.250547 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085062d8-c0f5-41b2-986c-ca5f4a99c27c None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:37.251112 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085062d8-c0f5-41b2-986c-ca5f4a99c27c None admin] PATH_INFO: `/v3/services` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:37.252317 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085062d8-c0f5-41b2-986c-ca5f4a99c27c None admin] RBAC: Adding query filter params (type=placement) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:37.252647 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085062d8-c0f5-41b2-986c-ca5f4a99c27c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:37.254189 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085062d8-c0f5-41b2-986c-ca5f4a99c27c None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:37.260785 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 490/969] 10.222.0.30 () {56 vars in 1116 bytes} [Wed Nov 2 07:18:37 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:39.430520 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338700df-40a6-4110-81e8-67eda01b09c4 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:39.431440 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338700df-40a6-4110-81e8-67eda01b09c4 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:39.431440 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338700df-40a6-4110-81e8-67eda01b09c4 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:39.433556 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 480/970] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:18:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:39.441573 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43782294-eb63-40f0-8125-78fe65f7476a None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:39.441975 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43782294-eb63-40f0-8125-78fe65f7476a None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:39.442116 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43782294-eb63-40f0-8125-78fe65f7476a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:39.463718 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-43782294-eb63-40f0-8125-78fe65f7476a None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:39.464906 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-43782294-eb63-40f0-8125-78fe65f7476a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:39.534806 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 491/971] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:18:39.799367 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338700df-40a6-4110-81e8-67eda01b09c4 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:39.913044 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338700df-40a6-4110-81e8-67eda01b09c4 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:39.946423 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad1337-9d39-41a6-9249-d82aff647b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:39.947045 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad1337-9d39-41a6-9249-d82aff647b6e None admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:39.947320 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad1337-9d39-41a6-9249-d82aff647b6e None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:39.947582 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad1337-9d39-41a6-9249-d82aff647b6e None admin] PATH_INFO: `/v3/services` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:39.948229 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad1337-9d39-41a6-9249-d82aff647b6e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:39.949595 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad1337-9d39-41a6-9249-d82aff647b6e None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:39.967246 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 481/972] 10.222.0.30 () {60 vars in 1141 bytes} [Wed Nov 2 07:18:39 2022] POST /identity/v3/services => generated 246 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:18:42.149930 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b9c8a-aaa9-4e52-866e-03a865bf75f7 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:42.150804 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b9c8a-aaa9-4e52-866e-03a865bf75f7 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:42.150804 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b9c8a-aaa9-4e52-866e-03a865bf75f7 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:42.153018 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 492/973] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:18:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:42.162917 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b358fa-3afb-4f9e-840e-44d8b931823f None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:42.163358 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b358fa-3afb-4f9e-840e-44d8b931823f None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:42.163480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b358fa-3afb-4f9e-840e-44d8b931823f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:42.186455 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-43b358fa-3afb-4f9e-840e-44d8b931823f None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:42.187692 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-43b358fa-3afb-4f9e-840e-44d8b931823f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:42.197389 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:42.268441 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 482/974] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:42 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:18:42.554393 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b9c8a-aaa9-4e52-866e-03a865bf75f7 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:42.658705 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:42.672306 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b9c8a-aaa9-4e52-866e-03a865bf75f7 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:42.709446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ccc0e-2402-4db8-af61-87da2603166d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:42.710076 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ccc0e-2402-4db8-af61-87da2603166d None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:42.710348 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ccc0e-2402-4db8-af61-87da2603166d None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:42.710615 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ccc0e-2402-4db8-af61-87da2603166d None admin] PATH_INFO: `/v3/services/placement` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:42.711226 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ccc0e-2402-4db8-af61-87da2603166d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:42.712594 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ccc0e-2402-4db8-af61-87da2603166d None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:42.721781 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-c01ccc0e-2402-4db8-af61-87da2603166d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:18:42.721781 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:42.721781 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:18:42.721781 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:18:42.721781 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:18:42.721781 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:18:42.721781 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:18:42.721781 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:18:42.721781 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:18:42.721781 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:18:42.721781 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:18:42.721781 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:18:42.721781 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:42.721781 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:42.721781 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:42.721781 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:42.721781 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:42.722990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:42.722990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:42.722990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:42.722990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:42.722990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:42.722990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:42.722990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:18:42.722990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:18:42.722990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:18:42.722990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:18:42.722990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:42.722990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:42.722990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:42.722990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:42.722990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:42.722990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:42.724428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:42.724428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:42.724428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:42.724428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:42.724428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:42.724428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:42.724428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:42.724428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:42.724428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:42.724428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:42.724428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:18:42.724428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:18:42.724428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:18:42.724428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:18:42.724428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 493/975] 10.222.0.30 () {56 vars in 1117 bytes} [Wed Nov 2 07:18:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:42.732377 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b358fa-3afb-4f9e-840e-44d8b931823f None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:42.838172 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b358fa-3afb-4f9e-840e-44d8b931823f None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:42.875882 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeede0e0-60af-44ed-a29c-fff859ec81dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:42.876477 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeede0e0-60af-44ed-a29c-fff859ec81dc None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:42.876745 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeede0e0-60af-44ed-a29c-fff859ec81dc None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:42.877005 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeede0e0-60af-44ed-a29c-fff859ec81dc None admin] PATH_INFO: `/v3/services` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:42.877643 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeede0e0-60af-44ed-a29c-fff859ec81dc None admin] RBAC: Adding query filter params (name=placement) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:42.877939 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeede0e0-60af-44ed-a29c-fff859ec81dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:42.879333 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeede0e0-60af-44ed-a29c-fff859ec81dc None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:42.884458 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 483/976] 10.222.0.30 () {56 vars in 1116 bytes} [Wed Nov 2 07:18:42 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:42.893663 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ccc0e-2402-4db8-af61-87da2603166d None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:43.007656 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ccc0e-2402-4db8-af61-87da2603166d None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:43.047767 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129a2e3-5424-4ff4-8046-fe6d639652a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:43.048360 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129a2e3-5424-4ff4-8046-fe6d639652a0 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:43.048632 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129a2e3-5424-4ff4-8046-fe6d639652a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:43.048897 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129a2e3-5424-4ff4-8046-fe6d639652a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:43.049637 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129a2e3-5424-4ff4-8046-fe6d639652a0 None admin] RBAC: Adding query filter params (interface=public, service_id=de725df209424acf8f8ae9e4249cb5b2, region_id=RegionOne) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:43.049936 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129a2e3-5424-4ff4-8046-fe6d639652a0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:43.051327 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129a2e3-5424-4ff4-8046-fe6d639652a0 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:43.056452 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 494/977] 10.222.0.30 () {56 vars in 1251 bytes} [Wed Nov 2 07:18:42 2022] GET /identity/v3/endpoints?service_id=de725df209424acf8f8ae9e4249cb5b2&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:43.066101 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeede0e0-60af-44ed-a29c-fff859ec81dc None admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:43.071984 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:43.186287 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeede0e0-60af-44ed-a29c-fff859ec81dc None admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:43.223381 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d838626-70ef-48bf-9c84-bbde92b232b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:43.223982 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d838626-70ef-48bf-9c84-bbde92b232b7 None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:43.224251 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d838626-70ef-48bf-9c84-bbde92b232b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:43.224513 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d838626-70ef-48bf-9c84-bbde92b232b7 None admin] PATH_INFO: `/v3/services` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:43.225106 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d838626-70ef-48bf-9c84-bbde92b232b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:43.226478 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d838626-70ef-48bf-9c84-bbde92b232b7 None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:43.233994 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 484/978] 10.222.0.30 () {56 vars in 1087 bytes} [Wed Nov 2 07:18:43 2022] GET /identity/v3/services => generated 2191 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:18:45.402759 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cfb8e8-c653-4860-a165-d59ec7c80984 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:45.403327 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cfb8e8-c653-4860-a165-d59ec7c80984 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:45.403327 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cfb8e8-c653-4860-a165-d59ec7c80984 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:45.404676 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 495/979] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:18:45.410183 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216ebb9-312f-4ed6-ac46-08c964e9264a None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:45.410617 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216ebb9-312f-4ed6-ac46-08c964e9264a None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:45.410753 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216ebb9-312f-4ed6-ac46-08c964e9264a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:45.431655 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-4216ebb9-312f-4ed6-ac46-08c964e9264a None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:18:45.432827 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-4216ebb9-312f-4ed6-ac46-08c964e9264a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:18:45.496357 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:45.502443 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 485/980] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:18:45 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 07:18:45.766861 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cfb8e8-c653-4860-a165-d59ec7c80984 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:45.790542 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:45.875608 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cfb8e8-c653-4860-a165-d59ec7c80984 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:45.912603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4f866-bf46-4673-84eb-07acf5c2e813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:45.913218 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4f866-bf46-4673-84eb-07acf5c2e813 None admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:45.913489 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4f866-bf46-4673-84eb-07acf5c2e813 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:45.913756 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4f866-bf46-4673-84eb-07acf5c2e813 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:45.914332 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f4f866-bf46-4673-84eb-07acf5c2e813 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:45.915707 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f4f866-bf46-4673-84eb-07acf5c2e813 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:45.924405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application [None req-61f4f866-bf46-4673-84eb-07acf5c2e813 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:18:45.924405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:45.924405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 07:18:45.924405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 07:18:45.924405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 07:18:45.924405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 07:18:45.924405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 07:18:45.924405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:18:45.924405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:18:45.924405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:18:45.924405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 07:18:45.924405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:18:45.924405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 07:18:45.924405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 07:18:45.924405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 07:18:45.924405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 07:18:45.924405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 07:18:45.925619 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 07:18:45.925619 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 07:18:45.925619 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 07:18:45.925619 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 07:18:45.925619 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 07:18:45.925619 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 07:18:45.925619 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 07:18:45.925619 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 07:18:45.925619 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 07:18:45.925619 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 07:18:45.925619 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 07:18:45.925619 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 07:18:45.925619 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 07:18:45.925619 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 07:18:45.925619 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 07:18:45.925619 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 07:18:45.926569 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 07:18:45.926569 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application with Lock( Nov 02 07:18:45.926569 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 07:18:45.926569 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self._enter() Nov 02 07:18:45.926569 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 07:18:45.926569 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 07:18:45.926569 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 07:18:45.926569 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application return self.creator() Nov 02 07:18:45.926569 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 07:18:45.926569 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application created_value = creator( Nov 02 07:18:45.926569 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 07:18:45.926569 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 07:18:45.926569 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 07:18:45.926569 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: ERROR keystone.server.flask.application Nov 02 07:18:45.927522 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 496/981] 10.222.0.30 () {56 vars in 1117 bytes} [Wed Nov 2 07:18:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 07:18:45.934557 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216ebb9-312f-4ed6-ac46-08c964e9264a None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:46.055664 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216ebb9-312f-4ed6-ac46-08c964e9264a None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:46.090436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e501c9-a4cd-4143-94e7-9e3475fcf7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:46.091062 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e501c9-a4cd-4143-94e7-9e3475fcf7bf None admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:46.091333 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e501c9-a4cd-4143-94e7-9e3475fcf7bf None admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:46.091593 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e501c9-a4cd-4143-94e7-9e3475fcf7bf None admin] PATH_INFO: `/v3/services` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:46.092198 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e501c9-a4cd-4143-94e7-9e3475fcf7bf None admin] RBAC: Adding query filter params (name=placement) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:18:46.092500 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e501c9-a4cd-4143-94e7-9e3475fcf7bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:46.093850 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e501c9-a4cd-4143-94e7-9e3475fcf7bf None admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:46.099137 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 486/982] 10.222.0.30 () {56 vars in 1116 bytes} [Wed Nov 2 07:18:45 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:18:46.108546 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4f866-bf46-4673-84eb-07acf5c2e813 None admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:18:46.197894 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:46.219009 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4f866-bf46-4673-84eb-07acf5c2e813 None admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:18:46.256790 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f38c6-9699-4692-b397-581f395e7840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:18:46.257393 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f38c6-9699-4692-b397-581f395e7840 None admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:18:46.257662 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f38c6-9699-4692-b397-581f395e7840 None admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:18:46.257930 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f38c6-9699-4692-b397-581f395e7840 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:18:46.258588 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f38c6-9699-4692-b397-581f395e7840 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:18:46.260076 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f38c6-9699-4692-b397-581f395e7840 None admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:18:46.282585 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 497/983] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:18:46 2022] POST /identity/v3/endpoints => generated 331 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:18:56.096482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=69 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:18:56.278102 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:50.623734 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd692ecb-0956-4c09-afc9-ffe93a676a61 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:50.624226 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd692ecb-0956-4c09-afc9-ffe93a676a61 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:50.624318 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd692ecb-0956-4c09-afc9-ffe93a676a61 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:50.629953 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 487/984] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:19:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:50.631573 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811abf0-fa73-4e8d-b97d-7cb58a03926a None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:50.631984 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811abf0-fa73-4e8d-b97d-7cb58a03926a None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:50.632106 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811abf0-fa73-4e8d-b97d-7cb58a03926a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:50.721697 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-f811abf0-fa73-4e8d-b97d-7cb58a03926a None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:50.723147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-f811abf0-fa73-4e8d-b97d-7cb58a03926a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:50.871273 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 498/985] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:19:51.152144 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c4820-4c92-409e-8e40-1afb52cff665 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:51.152481 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c4820-4c92-409e-8e40-1afb52cff665 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:51.152689 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c4820-4c92-409e-8e40-1afb52cff665 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:51.154267 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 488/986] 10.222.0.30 () {54 vars in 965 bytes} [Wed Nov 2 07:19:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:51.160225 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45e067-4193-4427-9f77-28ec29bda15b None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:51.160617 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45e067-4193-4427-9f77-28ec29bda15b None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:51.160984 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45e067-4193-4427-9f77-28ec29bda15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:51.243009 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-ed45e067-4193-4427-9f77-28ec29bda15b None None] MFA Rules not processed for user `65d51ec74e1e4c25bcc4365eb858bac0`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:51.244619 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-ed45e067-4193-4427-9f77-28ec29bda15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:51.268748 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:51.331816 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 499/987] 10.222.0.30 () {58 vars in 1062 bytes} [Wed Nov 2 07:19:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:19:51.341501 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70c4820-4c92-409e-8e40-1afb52cff665 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:51.343344 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-b70c4820-4c92-409e-8e40-1afb52cff665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:51.464951 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70c4820-4c92-409e-8e40-1afb52cff665 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:51.552301 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb98b096-a72a-4a21-b050-1fcd8628a090 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:51.552921 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98b096-a72a-4a21-b050-1fcd8628a090 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:51.553198 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98b096-a72a-4a21-b050-1fcd8628a090 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:51.553534 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98b096-a72a-4a21-b050-1fcd8628a090 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:51.555467 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-bb98b096-a72a-4a21-b050-1fcd8628a090 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:51.623115 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:51.696521 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb98b096-a72a-4a21-b050-1fcd8628a090 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:51.698900 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb98b096-a72a-4a21-b050-1fcd8628a090 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:51.722488 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 489/988] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:19:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:19:54.180627 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648614fd-2805-4383-96aa-eebfb6e72d65 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:54.181072 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648614fd-2805-4383-96aa-eebfb6e72d65 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:54.181164 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648614fd-2805-4383-96aa-eebfb6e72d65 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:54.182619 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 500/989] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:54.187897 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77b4f8-7cc5-4880-8aa5-1d882e8465ef None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:54.188311 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77b4f8-7cc5-4880-8aa5-1d882e8465ef None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:54.188437 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77b4f8-7cc5-4880-8aa5-1d882e8465ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:54.271198 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-ba77b4f8-7cc5-4880-8aa5-1d882e8465ef None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:54.272419 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-ba77b4f8-7cc5-4880-8aa5-1d882e8465ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:54.373580 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 490/990] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:19:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:19:54.670572 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137cfa7-fb8d-423d-82f3-186c22fcb507 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:54.670813 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137cfa7-fb8d-423d-82f3-186c22fcb507 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:54.671131 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137cfa7-fb8d-423d-82f3-186c22fcb507 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:54.673187 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 501/991] 10.222.0.30 () {54 vars in 965 bytes} [Wed Nov 2 07:19:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:54.681675 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314981d2-747f-49f9-95d5-23c5f57d668c None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:54.682139 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314981d2-747f-49f9-95d5-23c5f57d668c None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:54.682488 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314981d2-747f-49f9-95d5-23c5f57d668c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:54.700263 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:54.777456 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-314981d2-747f-49f9-95d5-23c5f57d668c None None] MFA Rules not processed for user `65d51ec74e1e4c25bcc4365eb858bac0`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:54.778685 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-314981d2-747f-49f9-95d5-23c5f57d668c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:54.855921 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 491/992] 10.222.0.30 () {58 vars in 1062 bytes} [Wed Nov 2 07:19:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:19:54.865599 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137cfa7-fb8d-423d-82f3-186c22fcb507 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:54.868094 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-2137cfa7-fb8d-423d-82f3-186c22fcb507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:54.984696 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137cfa7-fb8d-423d-82f3-186c22fcb507 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:55.070505 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:55.073519 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f15de8-b527-462c-8221-4504f47093a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:55.074335 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f15de8-b527-462c-8221-4504f47093a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:55.074671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f15de8-b527-462c-8221-4504f47093a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:55.075291 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f15de8-b527-462c-8221-4504f47093a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:55.077205 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-67f15de8-b527-462c-8221-4504f47093a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:55.189266 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f15de8-b527-462c-8221-4504f47093a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:55.190706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f15de8-b527-462c-8221-4504f47093a0 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:55.208610 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 502/993] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:19:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:19:57.907289 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc09ea5f-774f-4b73-b091-235dd73951dc None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:57.908144 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc09ea5f-774f-4b73-b091-235dd73951dc None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:57.908144 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc09ea5f-774f-4b73-b091-235dd73951dc None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:57.910695 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 492/994] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:19:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:19:57.918834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd95217-d7a1-4b37-97db-bfa05b9c64de None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:57.919279 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd95217-d7a1-4b37-97db-bfa05b9c64de None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:57.919392 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd95217-d7a1-4b37-97db-bfa05b9c64de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:57.999782 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-5dd95217-d7a1-4b37-97db-bfa05b9c64de None None] MFA Rules not processed for user `dc54984fb77c49da93c625769084ee80`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:19:58.000982 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-5dd95217-d7a1-4b37-97db-bfa05b9c64de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:58.059229 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:58.107403 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 503/995] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:19:57 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:19:58.395234 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc09ea5f-774f-4b73-b091-235dd73951dc None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:19:58.397631 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-fc09ea5f-774f-4b73-b091-235dd73951dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:58.509446 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc09ea5f-774f-4b73-b091-235dd73951dc None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:19:58.512355 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:19:58.588889 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0e915d-0a0e-4d32-b70d-261715dfae4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:19:58.589894 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e915d-0a0e-4d32-b70d-261715dfae4d service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:19:58.590169 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e915d-0a0e-4d32-b70d-261715dfae4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:19:58.590436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e915d-0a0e-4d32-b70d-261715dfae4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:19:58.592085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-5a0e915d-0a0e-4d32-b70d-261715dfae4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:19:58.717399 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0e915d-0a0e-4d32-b70d-261715dfae4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:19:58.718798 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0e915d-0a0e-4d32-b70d-261715dfae4d service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:19:58.737848 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 493/996] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:19:58 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:01.560532 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f094210-e9c7-4595-b2a3-b2d9e857f7ca None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:01.561104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f094210-e9c7-4595-b2a3-b2d9e857f7ca None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:01.561104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f094210-e9c7-4595-b2a3-b2d9e857f7ca None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:01.562411 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 504/997] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:01.568979 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f5cf5-265d-4c52-a47d-1a7ba22d6787 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:01.569742 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f5cf5-265d-4c52-a47d-1a7ba22d6787 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:01.569891 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f5cf5-265d-4c52-a47d-1a7ba22d6787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:01.620531 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:01.657574 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-716f5cf5-265d-4c52-a47d-1a7ba22d6787 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:01.658192 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-716f5cf5-265d-4c52-a47d-1a7ba22d6787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:01.759697 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 494/998] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:02.063658 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f094210-e9c7-4595-b2a3-b2d9e857f7ca None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:02.066150 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-2f094210-e9c7-4595-b2a3-b2d9e857f7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:02.188828 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f094210-e9c7-4595-b2a3-b2d9e857f7ca None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:02.266986 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2170a744-421e-474a-8a12-a4699ff4f2a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:02.267594 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170a744-421e-474a-8a12-a4699ff4f2a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:02.267870 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170a744-421e-474a-8a12-a4699ff4f2a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:02.268139 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170a744-421e-474a-8a12-a4699ff4f2a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:02.269727 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-2170a744-421e-474a-8a12-a4699ff4f2a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:02.277369 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:02.392913 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2170a744-421e-474a-8a12-a4699ff4f2a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:02.394358 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2170a744-421e-474a-8a12-a4699ff4f2a2 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:02.413162 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 505/999] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:05.257629 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc3a50d-74b2-4767-b5b0-920100027f95 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:05.258213 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc3a50d-74b2-4767-b5b0-920100027f95 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:05.258213 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc3a50d-74b2-4767-b5b0-920100027f95 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:05.259558 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 495/1000] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:20:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:05.265215 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a2717-7d89-47fb-8b2c-8a12c0d8254c None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:05.265713 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a2717-7d89-47fb-8b2c-8a12c0d8254c None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:05.265788 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a2717-7d89-47fb-8b2c-8a12c0d8254c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:05.343625 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-6c3a2717-7d89-47fb-8b2c-8a12c0d8254c None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:05.344871 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-6c3a2717-7d89-47fb-8b2c-8a12c0d8254c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:05.410007 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:05.454410 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 506/1001] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:05.732922 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc3a50d-74b2-4767-b5b0-920100027f95 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:05.734523 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-bfc3a50d-74b2-4767-b5b0-920100027f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:05.851387 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc3a50d-74b2-4767-b5b0-920100027f95 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:05.865969 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:05.930267 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34748d-1772-4a34-affe-9281f29f485d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:05.930912 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34748d-1772-4a34-affe-9281f29f485d service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:05.931188 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34748d-1772-4a34-affe-9281f29f485d service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:05.931455 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34748d-1772-4a34-affe-9281f29f485d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:05.933160 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-ac34748d-1772-4a34-affe-9281f29f485d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:06.047653 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34748d-1772-4a34-affe-9281f29f485d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:06.049116 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34748d-1772-4a34-affe-9281f29f485d service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:06.067732 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 496/1002] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:08.272324 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05824e-cbeb-4e2b-a0ee-6b2d38d0cc7b None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:08.272865 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05824e-cbeb-4e2b-a0ee-6b2d38d0cc7b None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:08.272865 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05824e-cbeb-4e2b-a0ee-6b2d38d0cc7b None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:08.274647 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 507/1003] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:20:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:08.280685 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311b6c0-b2bb-4c51-8380-17ae961bf46d None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:08.281159 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311b6c0-b2bb-4c51-8380-17ae961bf46d None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:08.281260 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311b6c0-b2bb-4c51-8380-17ae961bf46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:08.330306 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:08.348965 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-0311b6c0-b2bb-4c51-8380-17ae961bf46d None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:08.350251 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-0311b6c0-b2bb-4c51-8380-17ae961bf46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:08.460033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 497/1004] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:08.739923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05824e-cbeb-4e2b-a0ee-6b2d38d0cc7b None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:08.741561 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-1a05824e-cbeb-4e2b-a0ee-6b2d38d0cc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:08.849834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05824e-cbeb-4e2b-a0ee-6b2d38d0cc7b None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:08.929638 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22032e25-ea16-466a-84dd-7d8f7b51aa7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:08.930164 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22032e25-ea16-466a-84dd-7d8f7b51aa7c service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:08.930438 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22032e25-ea16-466a-84dd-7d8f7b51aa7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:08.930732 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22032e25-ea16-466a-84dd-7d8f7b51aa7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:08.932327 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-22032e25-ea16-466a-84dd-7d8f7b51aa7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:08.949640 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:09.045280 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22032e25-ea16-466a-84dd-7d8f7b51aa7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:09.046890 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22032e25-ea16-466a-84dd-7d8f7b51aa7c service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:09.066697 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 508/1005] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:11.286967 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae46fb-3210-492d-b433-2fd57c3e8ef0 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:11.287487 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae46fb-3210-492d-b433-2fd57c3e8ef0 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:11.287487 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae46fb-3210-492d-b433-2fd57c3e8ef0 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:11.288864 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 498/1006] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:11.295001 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6652b-a5bc-4099-9414-c30eeac2bb61 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:11.295445 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6652b-a5bc-4099-9414-c30eeac2bb61 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:11.295553 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6652b-a5bc-4099-9414-c30eeac2bb61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:11.368263 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-8cf6652b-a5bc-4099-9414-c30eeac2bb61 None None] MFA Rules not processed for user `dc54984fb77c49da93c625769084ee80`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:11.369479 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-8cf6652b-a5bc-4099-9414-c30eeac2bb61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:11.443293 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:11.477194 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 509/1007] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:11 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:20:11.767471 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae46fb-3210-492d-b433-2fd57c3e8ef0 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:11.769007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-9eae46fb-3210-492d-b433-2fd57c3e8ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:11.879761 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae46fb-3210-492d-b433-2fd57c3e8ef0 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:11.907805 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:11.966935 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c772f75-68ae-452d-9d56-a68730393a14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:11.967540 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c772f75-68ae-452d-9d56-a68730393a14 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:11.967812 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c772f75-68ae-452d-9d56-a68730393a14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:11.968076 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c772f75-68ae-452d-9d56-a68730393a14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:11.969653 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-9c772f75-68ae-452d-9d56-a68730393a14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:12.084055 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c772f75-68ae-452d-9d56-a68730393a14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:12.085410 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c772f75-68ae-452d-9d56-a68730393a14 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:12.105297 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 499/1008] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:11 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:14.479881 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc92e94-fdb6-49b3-ac3f-96f924a71297 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:14.480388 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc92e94-fdb6-49b3-ac3f-96f924a71297 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:14.480976 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc92e94-fdb6-49b3-ac3f-96f924a71297 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:14.481806 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 510/1009] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:20:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:14.488309 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c0e67-c8de-493e-813d-3b7d27b87c96 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:14.488750 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c0e67-c8de-493e-813d-3b7d27b87c96 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:14.488852 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c0e67-c8de-493e-813d-3b7d27b87c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:14.554732 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:14.558219 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-177c0e67-c8de-493e-813d-3b7d27b87c96 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:14.560012 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-177c0e67-c8de-493e-813d-3b7d27b87c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:14.663079 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 500/1010] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:14.950232 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc92e94-fdb6-49b3-ac3f-96f924a71297 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:14.951820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-1cc92e94-fdb6-49b3-ac3f-96f924a71297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:15.064600 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc92e94-fdb6-49b3-ac3f-96f924a71297 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:15.144306 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbc3185-ab79-49b0-9ca5-926fb0523f6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:15.144878 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc3185-ab79-49b0-9ca5-926fb0523f6b service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:15.145187 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc3185-ab79-49b0-9ca5-926fb0523f6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:15.145455 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc3185-ab79-49b0-9ca5-926fb0523f6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:15.147082 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-cdbc3185-ab79-49b0-9ca5-926fb0523f6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:15.176074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:15.265227 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbc3185-ab79-49b0-9ca5-926fb0523f6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:15.266618 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbc3185-ab79-49b0-9ca5-926fb0523f6b service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:15.285174 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 511/1011] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:17.462337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ba104-a554-44ae-851f-cea34612d1f7 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:17.463364 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ba104-a554-44ae-851f-cea34612d1f7 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:17.463364 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ba104-a554-44ae-851f-cea34612d1f7 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:17.465711 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 501/1012] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:20:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:17.471295 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191cd08d-7185-4c49-a4b8-47147b6fa989 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:17.471718 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191cd08d-7185-4c49-a4b8-47147b6fa989 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:17.471836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191cd08d-7185-4c49-a4b8-47147b6fa989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:17.547518 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-191cd08d-7185-4c49-a4b8-47147b6fa989 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:17.548740 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-191cd08d-7185-4c49-a4b8-47147b6fa989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:17.633477 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:17.657465 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 512/1013] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:17.943322 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ba104-a554-44ae-851f-cea34612d1f7 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:17.945802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-f08ba104-a554-44ae-851f-cea34612d1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:18.060816 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ba104-a554-44ae-851f-cea34612d1f7 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:18.089734 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:18.137309 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e20617-9a96-474d-8e39-2a3b815a20e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:18.137849 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e20617-9a96-474d-8e39-2a3b815a20e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:18.138121 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e20617-9a96-474d-8e39-2a3b815a20e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:18.138385 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e20617-9a96-474d-8e39-2a3b815a20e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:18.140005 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-22e20617-9a96-474d-8e39-2a3b815a20e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:18.263954 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e20617-9a96-474d-8e39-2a3b815a20e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:18.265367 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e20617-9a96-474d-8e39-2a3b815a20e0 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:18.284628 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 502/1014] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:20.892067 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4b787-295d-4af5-a71d-92183ab10798 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:20.892641 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4b787-295d-4af5-a71d-92183ab10798 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:20.892641 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4b787-295d-4af5-a71d-92183ab10798 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:20.894061 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 513/1015] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:20:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:20.901351 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c150f-13af-4919-9a68-e4daf5973d5c None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:20.902034 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c150f-13af-4919-9a68-e4daf5973d5c None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:20.902175 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c150f-13af-4919-9a68-e4daf5973d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:20.969740 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-7a6c150f-13af-4919-9a68-e4daf5973d5c None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:20.971004 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-7a6c150f-13af-4919-9a68-e4daf5973d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:20.979763 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:21.076077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 503/1016] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:21.365546 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4b787-295d-4af5-a71d-92183ab10798 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:21.368531 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-85d4b787-295d-4af5-a71d-92183ab10798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:21.493511 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4b787-295d-4af5-a71d-92183ab10798 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:21.577270 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b3a4c-1b37-4123-8f4f-6694d2ca7aef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:21.577862 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b3a4c-1b37-4123-8f4f-6694d2ca7aef service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:21.578150 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b3a4c-1b37-4123-8f4f-6694d2ca7aef service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:21.578439 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b3a4c-1b37-4123-8f4f-6694d2ca7aef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:21.580072 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-6e1b3a4c-1b37-4123-8f4f-6694d2ca7aef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:21.622887 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:21.700756 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1b3a4c-1b37-4123-8f4f-6694d2ca7aef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:21.702211 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1b3a4c-1b37-4123-8f4f-6694d2ca7aef service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:21.720966 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 514/1017] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:26.028424 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd4fd5-7d06-40d2-a574-0cf335b54ff8 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:26.029022 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd4fd5-7d06-40d2-a574-0cf335b54ff8 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:26.029022 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd4fd5-7d06-40d2-a574-0cf335b54ff8 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:26.030256 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 504/1018] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:20:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:26.035971 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cdc9bc-8b1b-46cb-83ed-3047810eda2b None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:26.036386 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cdc9bc-8b1b-46cb-83ed-3047810eda2b None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:26.036527 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cdc9bc-8b1b-46cb-83ed-3047810eda2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:26.105353 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-32cdc9bc-8b1b-46cb-83ed-3047810eda2b None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:26.106614 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-32cdc9bc-8b1b-46cb-83ed-3047810eda2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:26.193104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:26.205228 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 515/1019] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:26.484579 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd4fd5-7d06-40d2-a574-0cf335b54ff8 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:26.486128 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-65bd4fd5-7d06-40d2-a574-0cf335b54ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:26.596998 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd4fd5-7d06-40d2-a574-0cf335b54ff8 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:26.638979 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:26.677707 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05614cd4-156c-4ba5-beea-a3bd513538a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:26.678315 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05614cd4-156c-4ba5-beea-a3bd513538a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:26.678589 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05614cd4-156c-4ba5-beea-a3bd513538a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:26.679144 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05614cd4-156c-4ba5-beea-a3bd513538a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:26.680670 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-05614cd4-156c-4ba5-beea-a3bd513538a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:26.798859 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05614cd4-156c-4ba5-beea-a3bd513538a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:26.800308 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05614cd4-156c-4ba5-beea-a3bd513538a7 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:26.819514 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 505/1020] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:29.582795 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187efc8-f25a-4e95-a275-4b2f29466a6d None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:29.583327 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187efc8-f25a-4e95-a275-4b2f29466a6d None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:29.583327 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187efc8-f25a-4e95-a275-4b2f29466a6d None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:29.590110 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 516/1021] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:29.590274 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d2965-7d81-4ff1-a740-01ca266d322e None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:29.590274 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d2965-7d81-4ff1-a740-01ca266d322e None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:29.591245 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d2965-7d81-4ff1-a740-01ca266d322e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:29.660418 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-341d2965-7d81-4ff1-a740-01ca266d322e None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:29.661622 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-341d2965-7d81-4ff1-a740-01ca266d322e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:29.680079 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:29.765078 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 506/1022] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:30.045666 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187efc8-f25a-4e95-a275-4b2f29466a6d None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:30.047244 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-6187efc8-f25a-4e95-a275-4b2f29466a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:30.165004 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187efc8-f25a-4e95-a275-4b2f29466a6d None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:30.242010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e735f-c224-46ad-98b0-f4580c24362b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:30.242611 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e735f-c224-46ad-98b0-f4580c24362b service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:30.242916 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e735f-c224-46ad-98b0-f4580c24362b service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:30.243182 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e735f-c224-46ad-98b0-f4580c24362b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:30.244748 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-0b0e735f-c224-46ad-98b0-f4580c24362b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:30.296376 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:30.368120 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e735f-c224-46ad-98b0-f4580c24362b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:30.369561 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e735f-c224-46ad-98b0-f4580c24362b service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:30.388786 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 517/1023] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:20:34.803037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6ce00-6e9f-4ed2-a32e-b36a08aeead6 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:34.803581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6ce00-6e9f-4ed2-a32e-b36a08aeead6 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:34.803581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6ce00-6e9f-4ed2-a32e-b36a08aeead6 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:34.805006 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 507/1024] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:20:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:34.810550 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201232d-25a9-4fd4-955d-c39785eebd41 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:34.811133 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201232d-25a9-4fd4-955d-c39785eebd41 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:34.811133 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201232d-25a9-4fd4-955d-c39785eebd41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:34.886806 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-4201232d-25a9-4fd4-955d-c39785eebd41 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:34.888034 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-4201232d-25a9-4fd4-955d-c39785eebd41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:34.997312 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:34.999839 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 518/1025] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:35.284326 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a6ce00-6e9f-4ed2-a32e-b36a08aeead6 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:35.286740 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-30a6ce00-6e9f-4ed2-a32e-b36a08aeead6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:35.409283 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a6ce00-6e9f-4ed2-a32e-b36a08aeead6 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:35.459445 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:35.492321 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af03ef85-2262-4d0d-9e2b-dd0669325b2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:35.492934 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03ef85-2262-4d0d-9e2b-dd0669325b2f service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:35.493210 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03ef85-2262-4d0d-9e2b-dd0669325b2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:35.493466 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03ef85-2262-4d0d-9e2b-dd0669325b2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:35.495116 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-af03ef85-2262-4d0d-9e2b-dd0669325b2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:35.624246 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af03ef85-2262-4d0d-9e2b-dd0669325b2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:35.625692 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af03ef85-2262-4d0d-9e2b-dd0669325b2f service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:35.644983 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 508/1026] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:40.739698 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb94c8b-6c5d-4dba-b077-958aa591ef59 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:40.740196 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb94c8b-6c5d-4dba-b077-958aa591ef59 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:40.740277 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb94c8b-6c5d-4dba-b077-958aa591ef59 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:40.742041 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 519/1027] 10.222.0.30 () {54 vars in 921 bytes} [Wed Nov 2 07:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:40.747420 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88258d82-b8d4-4f5a-b36f-e29d53f72417 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:40.747816 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88258d82-b8d4-4f5a-b36f-e29d53f72417 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:40.748007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88258d82-b8d4-4f5a-b36f-e29d53f72417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:40.912166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-88258d82-b8d4-4f5a-b36f-e29d53f72417 None None] MFA Rules not processed for user `932d3076fb4b426d8c83d02eb60d8063`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:40.913835 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-88258d82-b8d4-4f5a-b36f-e29d53f72417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:40.935789 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:41.045571 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 509/1028] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Nov 2 07:20:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:45.120334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6825c-238b-43ad-970f-a4259f753373 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:45.120826 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6825c-238b-43ad-970f-a4259f753373 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:45.120917 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6825c-238b-43ad-970f-a4259f753373 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:45.122491 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 520/1029] 10.222.0.30 () {54 vars in 921 bytes} [Wed Nov 2 07:20:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:45.127635 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099c4e4-b279-4c2f-ac59-4315679af1f5 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:45.128078 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099c4e4-b279-4c2f-ac59-4315679af1f5 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:45.128162 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099c4e4-b279-4c2f-ac59-4315679af1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:45.206097 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-7099c4e4-b279-4c2f-ac59-4315679af1f5 None None] MFA Rules not processed for user `932d3076fb4b426d8c83d02eb60d8063`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:45.207403 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-7099c4e4-b279-4c2f-ac59-4315679af1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:45.283630 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:45.322345 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 510/1030] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Nov 2 07:20:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:46.755693 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74efa24f-2c6e-431c-bc5d-8c909040737a None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:46.756250 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74efa24f-2c6e-431c-bc5d-8c909040737a None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:46.756250 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74efa24f-2c6e-431c-bc5d-8c909040737a None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:46.757756 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 521/1031] 10.222.0.30 () {54 vars in 921 bytes} [Wed Nov 2 07:20:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:46.763026 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c149ba-1f2f-4f29-9bdc-4c9fa81f14bd None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:46.763481 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c149ba-1f2f-4f29-9bdc-4c9fa81f14bd None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:46.763577 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c149ba-1f2f-4f29-9bdc-4c9fa81f14bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:46.844474 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-50c149ba-1f2f-4f29-9bdc-4c9fa81f14bd None None] MFA Rules not processed for user `932d3076fb4b426d8c83d02eb60d8063`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:46.845714 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-50c149ba-1f2f-4f29-9bdc-4c9fa81f14bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:46.966009 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 511/1032] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Nov 2 07:20:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:50.328577 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b3a51-f1f6-4e8e-b24b-4dbca2f53b99 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:50.329506 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b3a51-f1f6-4e8e-b24b-4dbca2f53b99 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:50.329953 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b3a51-f1f6-4e8e-b24b-4dbca2f53b99 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:50.331778 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 522/1033] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:20:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:50.337627 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0754964-40bb-418d-a279-407a39995cf9 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:50.338430 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0754964-40bb-418d-a279-407a39995cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:50.338879 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0754964-40bb-418d-a279-407a39995cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:50.376851 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:50.412372 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-d0754964-40bb-418d-a279-407a39995cf9 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:50.413570 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-d0754964-40bb-418d-a279-407a39995cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:50.521277 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 512/1034] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:50.817258 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfff83-398b-4c30-b866-ab89a9b78017 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:50.817528 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfff83-398b-4c30-b866-ab89a9b78017 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:50.817816 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfff83-398b-4c30-b866-ab89a9b78017 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:50.819165 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 523/1035] 10.222.0.30 () {54 vars in 961 bytes} [Wed Nov 2 07:20:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:50.826578 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1a296-99a2-45c5-8ac9-448d90374eb1 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:50.826919 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1a296-99a2-45c5-8ac9-448d90374eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:50.827150 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1a296-99a2-45c5-8ac9-448d90374eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:50.941077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-71d1a296-99a2-45c5-8ac9-448d90374eb1 None None] MFA Rules not processed for user `e6495385dccc4fa48c05b1feb957846c`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:50.942279 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-71d1a296-99a2-45c5-8ac9-448d90374eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:50.993820 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:51.075185 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 513/1036] 10.222.0.30 () {58 vars in 1058 bytes} [Wed Nov 2 07:20:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:51.076423 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfff83-398b-4c30-b866-ab89a9b78017 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:51.078192 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-7bcfff83-398b-4c30-b866-ab89a9b78017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:51.232557 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfff83-398b-4c30-b866-ab89a9b78017 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:51.311501 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:51.353914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e9640-943c-4ca4-a4d1-cefc27bd94d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:51.354586 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e9640-943c-4ca4-a4d1-cefc27bd94d0 service nova] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:51.354896 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e9640-943c-4ca4-a4d1-cefc27bd94d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:51.355100 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e9640-943c-4ca4-a4d1-cefc27bd94d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:51.356682 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-c10e9640-943c-4ca4-a4d1-cefc27bd94d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:51.463621 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10e9640-943c-4ca4-a4d1-cefc27bd94d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:51.465390 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10e9640-943c-4ca4-a4d1-cefc27bd94d0 service nova] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:51.485265 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 524/1037] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:53.871397 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b18530-13d5-472c-8a5b-aca9c74e5813 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:53.871397 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b18530-13d5-472c-8a5b-aca9c74e5813 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:53.873905 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b18530-13d5-472c-8a5b-aca9c74e5813 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:53.877312 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 514/1038] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:20:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:53.878495 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af3218-54f2-4b67-8015-e5abe1cb4ae2 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:53.879098 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af3218-54f2-4b67-8015-e5abe1cb4ae2 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:53.879098 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af3218-54f2-4b67-8015-e5abe1cb4ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:53.948367 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-a1af3218-54f2-4b67-8015-e5abe1cb4ae2 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:53.949573 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-a1af3218-54f2-4b67-8015-e5abe1cb4ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:53.990463 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:54.054710 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 525/1039] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:20:54.342195 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b18530-13d5-472c-8a5b-aca9c74e5813 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:54.343767 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-76b18530-13d5-472c-8a5b-aca9c74e5813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:54.516916 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b18530-13d5-472c-8a5b-aca9c74e5813 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:54.529455 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:54.657062 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b118e5-6559-4cd3-bf48-03387716fea9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:54.657840 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b118e5-6559-4cd3-bf48-03387716fea9 service nova] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:54.657966 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b118e5-6559-4cd3-bf48-03387716fea9 service nova] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:54.658221 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b118e5-6559-4cd3-bf48-03387716fea9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:54.659882 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-37b118e5-6559-4cd3-bf48-03387716fea9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:54.750390 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b118e5-6559-4cd3-bf48-03387716fea9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:54.751978 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b118e5-6559-4cd3-bf48-03387716fea9 service nova] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:54.770685 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 515/1040] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:57.232866 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1e83d-8b41-497a-9c4e-d8dae30c9bd0 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:57.233798 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1e83d-8b41-497a-9c4e-d8dae30c9bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:57.234077 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1e83d-8b41-497a-9c4e-d8dae30c9bd0 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:57.235989 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 526/1041] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:20:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:57.241188 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26385c0-2f81-43fe-b7bf-26e10b81de23 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:57.241725 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26385c0-2f81-43fe-b7bf-26e10b81de23 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:57.241725 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26385c0-2f81-43fe-b7bf-26e10b81de23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:57.258613 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:57.316921 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-b26385c0-2f81-43fe-b7bf-26e10b81de23 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:57.318162 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-b26385c0-2f81-43fe-b7bf-26e10b81de23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:57.424088 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 516/1042] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:57.713888 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b2da1-0740-41ef-9baf-0f3ced7ed97a None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:57.714342 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b2da1-0740-41ef-9baf-0f3ced7ed97a None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:57.714795 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b2da1-0740-41ef-9baf-0f3ced7ed97a None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:57.716469 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 527/1043] 10.222.0.30 () {54 vars in 961 bytes} [Wed Nov 2 07:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:57.722876 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a9906-afa1-4ddc-9466-c5095a845822 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:57.723225 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a9906-afa1-4ddc-9466-c5095a845822 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:57.723436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a9906-afa1-4ddc-9466-c5095a845822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:57.804668 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-fb3a9906-afa1-4ddc-9466-c5095a845822 None None] MFA Rules not processed for user `e6495385dccc4fa48c05b1feb957846c`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:57.805883 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-fb3a9906-afa1-4ddc-9466-c5095a845822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:57.842503 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:57.930700 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 517/1044] 10.222.0.30 () {58 vars in 1058 bytes} [Wed Nov 2 07:20:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:57.938593 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b2da1-0740-41ef-9baf-0f3ced7ed97a None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:57.940188 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-4b4b2da1-0740-41ef-9baf-0f3ced7ed97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:58.100968 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b2da1-0740-41ef-9baf-0f3ced7ed97a None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:58.123121 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:58.226492 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea02644-40d5-47ab-872c-9e34eee43a18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:58.227041 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea02644-40d5-47ab-872c-9e34eee43a18 service nova] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:58.227320 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea02644-40d5-47ab-872c-9e34eee43a18 service nova] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:58.227590 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea02644-40d5-47ab-872c-9e34eee43a18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:58.229303 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-3ea02644-40d5-47ab-872c-9e34eee43a18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:58.332560 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea02644-40d5-47ab-872c-9e34eee43a18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:58.334389 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea02644-40d5-47ab-872c-9e34eee43a18 service nova] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:58.355411 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 528/1045] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:20:58.553925 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0640ef-d4ba-48ab-a6be-1b188d725cdb None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:58.554511 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0640ef-d4ba-48ab-a6be-1b188d725cdb None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:58.554511 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0640ef-d4ba-48ab-a6be-1b188d725cdb None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:58.556031 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 518/1046] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:58.561650 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35efa18b-bb20-4447-8a0a-3e10e5ec7b49 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:58.561973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35efa18b-bb20-4447-8a0a-3e10e5ec7b49 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:58.562159 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35efa18b-bb20-4447-8a0a-3e10e5ec7b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:58.602944 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:58.649269 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-35efa18b-bb20-4447-8a0a-3e10e5ec7b49 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:58.650741 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-35efa18b-bb20-4447-8a0a-3e10e5ec7b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:58.766686 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 529/1047] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:59.089051 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869254b-78b8-48f3-92b0-28e831a32478 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:59.089051 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869254b-78b8-48f3-92b0-28e831a32478 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:59.089357 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869254b-78b8-48f3-92b0-28e831a32478 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:59.091588 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 519/1048] 10.222.0.30 () {54 vars in 963 bytes} [Wed Nov 2 07:20:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:20:59.095663 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abfabde-57a3-4505-aced-4c13cafd5414 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:59.095925 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abfabde-57a3-4505-aced-4c13cafd5414 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:59.096226 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abfabde-57a3-4505-aced-4c13cafd5414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:59.177205 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-5abfabde-57a3-4505-aced-4c13cafd5414 None None] MFA Rules not processed for user `315861510b1041dca19d1e8753d9b21d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:20:59.179001 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-5abfabde-57a3-4505-aced-4c13cafd5414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:59.229000 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:59.263583 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 530/1049] 10.222.0.30 () {58 vars in 1060 bytes} [Wed Nov 2 07:20:59 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:20:59.271280 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869254b-78b8-48f3-92b0-28e831a32478 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:20:59.273049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-b869254b-78b8-48f3-92b0-28e831a32478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:59.395157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869254b-78b8-48f3-92b0-28e831a32478 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:20:59.445397 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:20:59.483223 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2569cebb-9ff0-4169-a806-a503947be8e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:20:59.483823 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569cebb-9ff0-4169-a806-a503947be8e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:20:59.484172 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569cebb-9ff0-4169-a806-a503947be8e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:20:59.484471 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569cebb-9ff0-4169-a806-a503947be8e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:20:59.486392 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-2569cebb-9ff0-4169-a806-a503947be8e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:20:59.635398 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2569cebb-9ff0-4169-a806-a503947be8e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:20:59.636757 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2569cebb-9ff0-4169-a806-a503947be8e1 service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:20:59.656349 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 520/1050] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:20:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:00.594728 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97403f01-7f65-4a2f-a6b3-48127e069a7a None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:00.595325 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97403f01-7f65-4a2f-a6b3-48127e069a7a None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:00.595325 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97403f01-7f65-4a2f-a6b3-48127e069a7a None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:00.596558 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 531/1051] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:00.602195 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c6a85-4285-422d-8a80-a03d5becc595 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:00.602443 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c6a85-4285-422d-8a80-a03d5becc595 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:00.602937 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c6a85-4285-422d-8a80-a03d5becc595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:00.672622 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-a04c6a85-4285-422d-8a80-a03d5becc595 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:00.673852 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-a04c6a85-4285-422d-8a80-a03d5becc595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:00.695511 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:00.771714 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 521/1052] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:21:01.057649 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97403f01-7f65-4a2f-a6b3-48127e069a7a None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:01.059352 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-97403f01-7f65-4a2f-a6b3-48127e069a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:01.221514 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97403f01-7f65-4a2f-a6b3-48127e069a7a None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:01.279284 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:01.347073 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf72fd8-74e6-4003-ac11-2df4bf8afc00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:01.347676 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf72fd8-74e6-4003-ac11-2df4bf8afc00 service nova] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:01.347949 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf72fd8-74e6-4003-ac11-2df4bf8afc00 service nova] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:01.348216 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf72fd8-74e6-4003-ac11-2df4bf8afc00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:01.349815 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-1cf72fd8-74e6-4003-ac11-2df4bf8afc00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:01.444888 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf72fd8-74e6-4003-ac11-2df4bf8afc00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:01.446530 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf72fd8-74e6-4003-ac11-2df4bf8afc00 service nova] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:01.465713 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 532/1053] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:03.692152 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bccbf-cc83-485e-bcea-cc26d38f6245 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:03.692750 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bccbf-cc83-485e-bcea-cc26d38f6245 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:03.692750 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bccbf-cc83-485e-bcea-cc26d38f6245 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:03.694196 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 522/1054] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:03.699477 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80aa4d-d0cb-4da9-a114-f882b6152cb9 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:03.699932 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80aa4d-d0cb-4da9-a114-f882b6152cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:03.700057 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80aa4d-d0cb-4da9-a114-f882b6152cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:03.769649 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-8a80aa4d-d0cb-4da9-a114-f882b6152cb9 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:03.770903 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-8a80aa4d-d0cb-4da9-a114-f882b6152cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:03.778695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:03.875441 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 533/1055] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:04.163015 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741bccbf-cc83-485e-bcea-cc26d38f6245 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:04.164803 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-741bccbf-cc83-485e-bcea-cc26d38f6245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:04.318733 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741bccbf-cc83-485e-bcea-cc26d38f6245 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:04.322883 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:04.437695 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193040a-94a8-4a63-b0dc-04d1ce4ccd66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:04.438196 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193040a-94a8-4a63-b0dc-04d1ce4ccd66 service nova] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:04.438461 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193040a-94a8-4a63-b0dc-04d1ce4ccd66 service nova] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:04.438761 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193040a-94a8-4a63-b0dc-04d1ce4ccd66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:04.440746 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-a193040a-94a8-4a63-b0dc-04d1ce4ccd66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:04.533908 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a193040a-94a8-4a63-b0dc-04d1ce4ccd66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:04.535520 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a193040a-94a8-4a63-b0dc-04d1ce4ccd66 service nova] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:04.554110 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 523/1056] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:06.742651 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf6e2b-f0f3-4d07-9b58-c4903ad1ef89 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:06.743251 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf6e2b-f0f3-4d07-9b58-c4903ad1ef89 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:06.743251 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf6e2b-f0f3-4d07-9b58-c4903ad1ef89 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:06.744647 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 534/1057] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:06.749917 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dabfc5-6a46-4d0b-99ae-be4d5b80dcf7 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:06.750356 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dabfc5-6a46-4d0b-99ae-be4d5b80dcf7 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:06.750436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dabfc5-6a46-4d0b-99ae-be4d5b80dcf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:06.769037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:06.828695 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-82dabfc5-6a46-4d0b-99ae-be4d5b80dcf7 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:06.829974 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-82dabfc5-6a46-4d0b-99ae-be4d5b80dcf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:06.931394 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 524/1058] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:07.211319 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf6e2b-f0f3-4d07-9b58-c4903ad1ef89 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:07.212907 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-5caf6e2b-f0f3-4d07-9b58-c4903ad1ef89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:07.360817 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:07.373606 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf6e2b-f0f3-4d07-9b58-c4903ad1ef89 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:07.502723 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa332581-d84a-4e86-a0b0-7502ee287f11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:07.503445 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa332581-d84a-4e86-a0b0-7502ee287f11 service nova] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:07.503732 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa332581-d84a-4e86-a0b0-7502ee287f11 service nova] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:07.504010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa332581-d84a-4e86-a0b0-7502ee287f11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:07.505691 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-fa332581-d84a-4e86-a0b0-7502ee287f11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:07.610451 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa332581-d84a-4e86-a0b0-7502ee287f11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:07.612770 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa332581-d84a-4e86-a0b0-7502ee287f11 service nova] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:07.625345 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:07.636457 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 535/1059] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:09.856715 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544a5a6-38f2-46b8-9283-d85bef40745d None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:09.857720 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544a5a6-38f2-46b8-9283-d85bef40745d None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:09.857720 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544a5a6-38f2-46b8-9283-d85bef40745d None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:09.858624 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 525/1060] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:21:09.864228 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7bb31-8815-4077-8be1-b8e1dc167547 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:09.864675 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7bb31-8815-4077-8be1-b8e1dc167547 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:09.864766 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7bb31-8815-4077-8be1-b8e1dc167547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:09.937620 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-46e7bb31-8815-4077-8be1-b8e1dc167547 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:09.938964 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-46e7bb31-8815-4077-8be1-b8e1dc167547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:10.064813 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 536/1061] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:10.346285 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544a5a6-38f2-46b8-9283-d85bef40745d None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:10.347829 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-1544a5a6-38f2-46b8-9283-d85bef40745d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:10.444512 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:10.507535 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544a5a6-38f2-46b8-9283-d85bef40745d None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:10.625271 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca54ae-759e-428c-8e15-d46ad11ec9b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:10.625924 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca54ae-759e-428c-8e15-d46ad11ec9b7 service nova] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:10.626186 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca54ae-759e-428c-8e15-d46ad11ec9b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:10.626420 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca54ae-759e-428c-8e15-d46ad11ec9b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:10.628171 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-17ca54ae-759e-428c-8e15-d46ad11ec9b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:10.685311 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:10.725591 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca54ae-759e-428c-8e15-d46ad11ec9b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:10.727329 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca54ae-759e-428c-8e15-d46ad11ec9b7 service nova] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:10.747662 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 526/1062] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:12.963999 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5191e97-6ae6-472e-83a9-de9ce17440f9 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:12.964521 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5191e97-6ae6-472e-83a9-de9ce17440f9 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:12.964521 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5191e97-6ae6-472e-83a9-de9ce17440f9 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:12.965814 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 537/1063] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:12.971854 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950056f9-93ec-43af-a2b0-155645cc0979 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:12.972426 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950056f9-93ec-43af-a2b0-155645cc0979 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:12.972426 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950056f9-93ec-43af-a2b0-155645cc0979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:13.039755 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-950056f9-93ec-43af-a2b0-155645cc0979 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:13.040967 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-950056f9-93ec-43af-a2b0-155645cc0979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:13.157362 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 527/1064] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:21:13.438423 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5191e97-6ae6-472e-83a9-de9ce17440f9 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:13.440866 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-c5191e97-6ae6-472e-83a9-de9ce17440f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:13.521202 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:13.596850 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5191e97-6ae6-472e-83a9-de9ce17440f9 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:13.714195 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e6eb9-4935-4d18-b8e0-7b4ca194c4f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:13.714670 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e6eb9-4935-4d18-b8e0-7b4ca194c4f4 service nova] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:13.714980 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e6eb9-4935-4d18-b8e0-7b4ca194c4f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:13.715307 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e6eb9-4935-4d18-b8e0-7b4ca194c4f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:13.716896 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-7c7e6eb9-4935-4d18-b8e0-7b4ca194c4f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:13.756584 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:13.810766 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7e6eb9-4935-4d18-b8e0-7b4ca194c4f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:13.812378 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7e6eb9-4935-4d18-b8e0-7b4ca194c4f4 service nova] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:13.831191 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 538/1065] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:14.895629 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655f81e-b47c-44b2-ba36-4539c1243711 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:14.896237 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655f81e-b47c-44b2-ba36-4539c1243711 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:14.896237 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655f81e-b47c-44b2-ba36-4539c1243711 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:14.897484 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 528/1066] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:14.903050 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee06b9c-8d23-4f2e-86d2-7f8aae224e9d None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:14.903827 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee06b9c-8d23-4f2e-86d2-7f8aae224e9d None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:14.903933 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee06b9c-8d23-4f2e-86d2-7f8aae224e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:14.978183 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-7ee06b9c-8d23-4f2e-86d2-7f8aae224e9d None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:14.979417 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-7ee06b9c-8d23-4f2e-86d2-7f8aae224e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:15.080034 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 539/1067] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:21:15.375614 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebf287-7417-4477-976b-2448224b1f0f None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:15.375864 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebf287-7417-4477-976b-2448224b1f0f None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:15.376156 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebf287-7417-4477-976b-2448224b1f0f None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:15.377518 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 529/1068] 10.222.0.30 () {54 vars in 962 bytes} [Wed Nov 2 07:21:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:15.382868 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c447ab9-5dba-46dd-8f0b-fdca619096e2 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:15.383266 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c447ab9-5dba-46dd-8f0b-fdca619096e2 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:15.383692 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c447ab9-5dba-46dd-8f0b-fdca619096e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:15.402412 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:15.464887 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-2c447ab9-5dba-46dd-8f0b-fdca619096e2 None None] MFA Rules not processed for user `9a051ca8154540ad93db6d028b8323a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:15.466070 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-2c447ab9-5dba-46dd-8f0b-fdca619096e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:15.544294 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 540/1069] 10.222.0.30 () {58 vars in 1059 bytes} [Wed Nov 2 07:21:15 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:15.552454 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebf287-7417-4477-976b-2448224b1f0f None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:15.553974 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-82ebf287-7417-4477-976b-2448224b1f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:15.587653 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:15.678987 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebf287-7417-4477-976b-2448224b1f0f None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:15.757575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c6085-5df6-4dd9-b053-7d69d953d78e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a051ca8154540ad93db6d028b8323a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:15.758186 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c6085-5df6-4dd9-b053-7d69d953d78e service glance] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:15.758463 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c6085-5df6-4dd9-b053-7d69d953d78e service glance] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:15.758738 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c6085-5df6-4dd9-b053-7d69d953d78e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:15.760366 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-f74c6085-5df6-4dd9-b053-7d69d953d78e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:15.840845 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:15.873892 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c6085-5df6-4dd9-b053-7d69d953d78e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:15.875417 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c6085-5df6-4dd9-b053-7d69d953d78e service glance] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:15.895074 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 530/1070] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:16.023474 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668cf33-31fa-4954-9b60-b096aed27f3e None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:16.024068 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668cf33-31fa-4954-9b60-b096aed27f3e None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:16.024068 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668cf33-31fa-4954-9b60-b096aed27f3e None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:16.025333 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 541/1071] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:16.031412 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba887593-4390-4cab-bdb1-cf8f72e759e5 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:16.031887 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba887593-4390-4cab-bdb1-cf8f72e759e5 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:16.031995 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba887593-4390-4cab-bdb1-cf8f72e759e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:16.110958 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-ba887593-4390-4cab-bdb1-cf8f72e759e5 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:16.113023 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-ba887593-4390-4cab-bdb1-cf8f72e759e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:16.218421 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 531/1072] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:16.508893 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668cf33-31fa-4954-9b60-b096aed27f3e None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:16.510532 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-9668cf33-31fa-4954-9b60-b096aed27f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:16.632421 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:16.655300 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb03204-7625-436d-90ae-a1c3c67204d6 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:16.655671 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb03204-7625-436d-90ae-a1c3c67204d6 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:16.656025 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb03204-7625-436d-90ae-a1c3c67204d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:16.680107 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668cf33-31fa-4954-9b60-b096aed27f3e None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:16.701769 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:16.724622 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-5cb03204-7625-436d-90ae-a1c3c67204d6 None None] MFA Rules not processed for user `364555ec385c4dbb91907fb9c5116d2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:16.725826 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-5cb03204-7625-436d-90ae-a1c3c67204d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:16.797604 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c045d0-fe62-4ed5-98d6-3ce61687368b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:16.798218 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c045d0-fe62-4ed5-98d6-3ce61687368b service nova] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:16.798492 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c045d0-fe62-4ed5-98d6-3ce61687368b service nova] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:16.798791 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c045d0-fe62-4ed5-98d6-3ce61687368b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:16.800445 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-e9c045d0-fe62-4ed5-98d6-3ce61687368b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:16.813055 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 532/1073] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Nov 2 07:21:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:21:16.860027 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae687dbb-9d45-4d30-9bea-ed2944dc2967 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:16.860328 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae687dbb-9d45-4d30-9bea-ed2944dc2967 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:16.860618 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae687dbb-9d45-4d30-9bea-ed2944dc2967 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:16.862018 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 533/1074] 10.222.0.30 () {54 vars in 943 bytes} [Wed Nov 2 07:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:16.868681 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0023a38-8a59-498b-b3e3-87693e89306e None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:16.868961 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0023a38-8a59-498b-b3e3-87693e89306e None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:16.869235 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0023a38-8a59-498b-b3e3-87693e89306e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:16.879999 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:16.907666 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c045d0-fe62-4ed5-98d6-3ce61687368b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:16.909321 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c045d0-fe62-4ed5-98d6-3ce61687368b service nova] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:16.927320 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 542/1075] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:21:16.949957 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-e0023a38-8a59-498b-b3e3-87693e89306e None None] MFA Rules not processed for user `836c4044aaf74de391219289b57d2956`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:16.951283 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-e0023a38-8a59-498b-b3e3-87693e89306e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:17.018947 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:17.073076 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 534/1076] 10.222.0.30 () {58 vars in 1040 bytes} [Wed Nov 2 07:21:16 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:21:17.083054 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c045d0-fe62-4ed5-98d6-3ce61687368b service nova] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:17.084304 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-e9c045d0-fe62-4ed5-98d6-3ce61687368b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:17.182931 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c045d0-fe62-4ed5-98d6-3ce61687368b service nova] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:17.286690 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:17.288338 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5903ae-df88-439a-a40c-348d93bcb23d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '836c4044aaf74de391219289b57d2956', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:17.288953 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5903ae-df88-439a-a40c-348d93bcb23d service swift] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:17.289220 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5903ae-df88-439a-a40c-348d93bcb23d service swift] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:17.289481 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5903ae-df88-439a-a40c-348d93bcb23d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:17.291775 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-5d5903ae-df88-439a-a40c-348d93bcb23d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:17.376899 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5903ae-df88-439a-a40c-348d93bcb23d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:17.378675 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5903ae-df88-439a-a40c-348d93bcb23d service swift] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:17.387062 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 543/1077] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Nov 2 07:21:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:21:19.159737 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a753e00-a5ce-43ff-b451-0ebb1e024d54 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:19.160354 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a753e00-a5ce-43ff-b451-0ebb1e024d54 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:19.160354 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a753e00-a5ce-43ff-b451-0ebb1e024d54 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:19.161647 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 535/1078] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:19.167165 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9a67c-7554-404e-b127-53274de443c3 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:19.167926 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9a67c-7554-404e-b127-53274de443c3 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:19.168369 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9a67c-7554-404e-b127-53274de443c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:19.242108 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-2cf9a67c-7554-404e-b127-53274de443c3 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:19.243368 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-2cf9a67c-7554-404e-b127-53274de443c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:19.346857 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:19.356044 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 544/1079] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:19.648628 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a753e00-a5ce-43ff-b451-0ebb1e024d54 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:19.650246 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-6a753e00-a5ce-43ff-b451-0ebb1e024d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:19.815864 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a753e00-a5ce-43ff-b451-0ebb1e024d54 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:19.864269 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:19.945057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0254f687-ac61-45f3-a776-c3192545ed36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:19.945673 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254f687-ac61-45f3-a776-c3192545ed36 service nova] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:19.945943 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254f687-ac61-45f3-a776-c3192545ed36 service nova] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:19.946210 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254f687-ac61-45f3-a776-c3192545ed36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:19.947839 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-0254f687-ac61-45f3-a776-c3192545ed36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:20.048650 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0254f687-ac61-45f3-a776-c3192545ed36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:20.050673 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0254f687-ac61-45f3-a776-c3192545ed36 service nova] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:20.070033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 536/1080] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:22.323658 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95494309-1ed7-430a-b67e-6476fea919f8 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:22.324261 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95494309-1ed7-430a-b67e-6476fea919f8 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:22.324261 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95494309-1ed7-430a-b67e-6476fea919f8 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:22.330061 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 545/1081] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:21:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:22.332241 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357f2f7-aa54-4027-bc22-c67cc8575677 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:22.332733 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357f2f7-aa54-4027-bc22-c67cc8575677 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:22.332852 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357f2f7-aa54-4027-bc22-c67cc8575677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:22.404133 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:22.415300 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-1357f2f7-aa54-4027-bc22-c67cc8575677 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:22.416650 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-1357f2f7-aa54-4027-bc22-c67cc8575677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:22.534706 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 537/1082] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:21:22.824852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95494309-1ed7-430a-b67e-6476fea919f8 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:22.826381 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-95494309-1ed7-430a-b67e-6476fea919f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:22.998790 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95494309-1ed7-430a-b67e-6476fea919f8 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:23.077851 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:23.133880 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2912f-c307-4a75-a7c3-4f03dfa93f97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:23.134615 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2912f-c307-4a75-a7c3-4f03dfa93f97 service nova] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:23.134972 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2912f-c307-4a75-a7c3-4f03dfa93f97 service nova] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:23.135301 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2912f-c307-4a75-a7c3-4f03dfa93f97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:23.136977 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-dcd2912f-c307-4a75-a7c3-4f03dfa93f97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:23.242526 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd2912f-c307-4a75-a7c3-4f03dfa93f97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:23.245370 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd2912f-c307-4a75-a7c3-4f03dfa93f97 service nova] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:23.280263 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 546/1083] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:25.557622 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e36787-6ec9-45f0-85f7-7cd3e0eee798 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:25.558211 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e36787-6ec9-45f0-85f7-7cd3e0eee798 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:25.558211 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e36787-6ec9-45f0-85f7-7cd3e0eee798 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:25.559520 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 538/1084] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:21:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:25.564809 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471d24f-a7b9-4cf6-a70c-06422c358ed8 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:25.565268 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471d24f-a7b9-4cf6-a70c-06422c358ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:25.565383 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471d24f-a7b9-4cf6-a70c-06422c358ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:25.645526 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-d471d24f-a7b9-4cf6-a70c-06422c358ed8 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:25.647627 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-d471d24f-a7b9-4cf6-a70c-06422c358ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:25.681607 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:25.762693 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 547/1085] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:21:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:26.046352 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e36787-6ec9-45f0-85f7-7cd3e0eee798 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:26.048103 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-c6e36787-6ec9-45f0-85f7-7cd3e0eee798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:26.178280 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:26.207162 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e36787-6ec9-45f0-85f7-7cd3e0eee798 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:26.328806 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba64838a-43c9-4adb-8e40-a66678d3d14f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:26.329421 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba64838a-43c9-4adb-8e40-a66678d3d14f service nova] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:26.329763 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba64838a-43c9-4adb-8e40-a66678d3d14f service nova] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:26.330039 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba64838a-43c9-4adb-8e40-a66678d3d14f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:26.331718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-ba64838a-43c9-4adb-8e40-a66678d3d14f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:26.426913 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:26.430794 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba64838a-43c9-4adb-8e40-a66678d3d14f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:26.432885 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba64838a-43c9-4adb-8e40-a66678d3d14f service nova] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:26.452930 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 539/1086] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:35.756992 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=30 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:36.451390 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=7 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:40.201308 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589fde4-3c66-4155-bdc1-3123e6b81ab3 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:40.201893 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589fde4-3c66-4155-bdc1-3123e6b81ab3 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:40.201893 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589fde4-3c66-4155-bdc1-3123e6b81ab3 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:40.203260 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 548/1087] 10.222.0.30 () {54 vars in 918 bytes} [Wed Nov 2 07:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:40.209524 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ec42f-b65a-463d-9a37-cb8cd2398bce None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:40.209975 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ec42f-b65a-463d-9a37-cb8cd2398bce None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:40.210077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ec42f-b65a-463d-9a37-cb8cd2398bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:40.307962 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-fd3ec42f-b65a-463d-9a37-cb8cd2398bce None None] MFA Rules not processed for user `932d3076fb4b426d8c83d02eb60d8063`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:40.309182 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-fd3ec42f-b65a-463d-9a37-cb8cd2398bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:40.432329 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 540/1088] 10.222.0.30 () {58 vars in 1015 bytes} [Wed Nov 2 07:21:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:40.450733 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f55bb0-70f3-4e67-a650-d8299b5bf5bc None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:40.451021 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f55bb0-70f3-4e67-a650-d8299b5bf5bc None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:40.451292 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f55bb0-70f3-4e67-a650-d8299b5bf5bc None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:40.452685 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 549/1089] 10.222.0.30 () {54 vars in 918 bytes} [Wed Nov 2 07:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:40.457458 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc5881-fa92-4799-8e12-958a5f892e26 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:40.457744 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc5881-fa92-4799-8e12-958a5f892e26 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:40.458011 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc5881-fa92-4799-8e12-958a5f892e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:40.534554 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:40.538322 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-d4cc5881-fa92-4799-8e12-958a5f892e26 None None] MFA Rules not processed for user `e6495385dccc4fa48c05b1feb957846c`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:40.540437 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-d4cc5881-fa92-4799-8e12-958a5f892e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:40.662844 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 541/1090] 10.222.0.30 () {58 vars in 1015 bytes} [Wed Nov 2 07:21:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:21:43.272539 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc1594-e5ca-40ba-bee9-707f1659a476 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:43.273127 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc1594-e5ca-40ba-bee9-707f1659a476 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:43.273127 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc1594-e5ca-40ba-bee9-707f1659a476 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:43.274412 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 550/1091] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:43.281264 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42f88a-444b-4099-8dcf-7d35b14597e5 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:43.281997 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42f88a-444b-4099-8dcf-7d35b14597e5 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:43.282179 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42f88a-444b-4099-8dcf-7d35b14597e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:43.356844 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-7a42f88a-444b-4099-8dcf-7d35b14597e5 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:43.358062 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-7a42f88a-444b-4099-8dcf-7d35b14597e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:43.417767 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:43.472469 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 542/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:43.766874 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fc1594-e5ca-40ba-bee9-707f1659a476 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:43.768409 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-a9fc1594-e5ca-40ba-bee9-707f1659a476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:43.934001 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fc1594-e5ca-40ba-bee9-707f1659a476 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:44.015617 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:44.058548 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400d8896-30d6-478f-806a-c389aaf4d007 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:44.059196 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d8896-30d6-478f-806a-c389aaf4d007 service nova] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:44.059472 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d8896-30d6-478f-806a-c389aaf4d007 service nova] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:44.059741 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d8896-30d6-478f-806a-c389aaf4d007 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:44.061331 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-400d8896-30d6-478f-806a-c389aaf4d007 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:44.153090 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400d8896-30d6-478f-806a-c389aaf4d007 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:44.154700 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400d8896-30d6-478f-806a-c389aaf4d007 service nova] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:44.172949 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 551/1093] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:45.303494 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c280f-7cf7-4d3c-9a61-bfe046cc570b None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:45.304097 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c280f-7cf7-4d3c-9a61-bfe046cc570b None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:45.304097 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c280f-7cf7-4d3c-9a61-bfe046cc570b None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:45.305576 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 543/1094] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:45.311952 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c68851-d5d3-4aa9-a268-f5671d119289 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:45.312661 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c68851-d5d3-4aa9-a268-f5671d119289 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:45.312852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c68851-d5d3-4aa9-a268-f5671d119289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:45.401252 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-d5c68851-d5d3-4aa9-a268-f5671d119289 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:45.402458 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-d5c68851-d5d3-4aa9-a268-f5671d119289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:45.430292 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:45.499799 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 552/1095] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:21:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:45.663939 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c280f-7cf7-4d3c-9a61-bfe046cc570b None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:45.666270 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-591c280f-7cf7-4d3c-9a61-bfe046cc570b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:45.830878 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c280f-7cf7-4d3c-9a61-bfe046cc570b None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:45.872591 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:45.957280 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e876d-0d38-4a02-8d19-e9129f50c889 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:45.957893 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e876d-0d38-4a02-8d19-e9129f50c889 service nova] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:45.958273 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e876d-0d38-4a02-8d19-e9129f50c889 service nova] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:45.958682 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e876d-0d38-4a02-8d19-e9129f50c889 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:45.961257 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-a07e876d-0d38-4a02-8d19-e9129f50c889 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:46.067515 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07e876d-0d38-4a02-8d19-e9129f50c889 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:46.069563 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07e876d-0d38-4a02-8d19-e9129f50c889 service nova] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:46.088188 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 544/1096] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:47.231364 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a07a8-da18-46ed-a274-8221026367a6 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:47.232303 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a07a8-da18-46ed-a274-8221026367a6 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:47.232303 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a07a8-da18-46ed-a274-8221026367a6 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:47.234389 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 553/1097] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:21:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:47.239528 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf567947-b229-43ee-a12b-09eb60785b73 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:47.239968 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf567947-b229-43ee-a12b-09eb60785b73 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:47.240380 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf567947-b229-43ee-a12b-09eb60785b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:47.299984 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:47.327678 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-cf567947-b229-43ee-a12b-09eb60785b73 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:47.328890 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-cf567947-b229-43ee-a12b-09eb60785b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:47.446124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 545/1098] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:21:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:47.602493 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2a07a8-da18-46ed-a274-8221026367a6 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:47.604166 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-1a2a07a8-da18-46ed-a274-8221026367a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:47.758761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2a07a8-da18-46ed-a274-8221026367a6 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:47.771662 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:47.881085 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a617f500-4deb-44b5-b903-0e6c47450c45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:47.881704 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617f500-4deb-44b5-b903-0e6c47450c45 service nova] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:47.881977 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617f500-4deb-44b5-b903-0e6c47450c45 service nova] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:47.882247 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617f500-4deb-44b5-b903-0e6c47450c45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:47.883878 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-a617f500-4deb-44b5-b903-0e6c47450c45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:47.976209 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a617f500-4deb-44b5-b903-0e6c47450c45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:47.977812 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a617f500-4deb-44b5-b903-0e6c47450c45 service nova] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:47.996579 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 554/1099] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:49.082389 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498ed7c-6a85-4c02-8831-3c31a5371f37 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:49.083018 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498ed7c-6a85-4c02-8831-3c31a5371f37 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:49.083018 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498ed7c-6a85-4c02-8831-3c31a5371f37 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:49.084375 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 546/1100] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:21:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:49.089464 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c61da-713b-492b-8877-514ecf673cf8 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:49.089910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c61da-713b-492b-8877-514ecf673cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:49.090006 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c61da-713b-492b-8877-514ecf673cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:49.111678 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:49.173234 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-6e8c61da-713b-492b-8877-514ecf673cf8 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:49.174430 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-6e8c61da-713b-492b-8877-514ecf673cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:49.280677 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 555/1101] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:21:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:49.437609 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5498ed7c-6a85-4c02-8831-3c31a5371f37 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:49.440133 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-5498ed7c-6a85-4c02-8831-3c31a5371f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:49.559035 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:49.597060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5498ed7c-6a85-4c02-8831-3c31a5371f37 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:49.716218 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8186426-414f-4efe-997f-cdf5b7bc4f88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:49.716828 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8186426-414f-4efe-997f-cdf5b7bc4f88 service nova] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:49.717103 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8186426-414f-4efe-997f-cdf5b7bc4f88 service nova] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:49.717361 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8186426-414f-4efe-997f-cdf5b7bc4f88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:49.718998 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-b8186426-414f-4efe-997f-cdf5b7bc4f88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:49.797313 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:49.812248 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8186426-414f-4efe-997f-cdf5b7bc4f88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:49.813773 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8186426-414f-4efe-997f-cdf5b7bc4f88 service nova] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:49.832009 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 547/1102] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:50.937667 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf924b0-9aae-4371-8207-9cf89aad6871 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:50.938200 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf924b0-9aae-4371-8207-9cf89aad6871 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:50.938200 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf924b0-9aae-4371-8207-9cf89aad6871 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:50.939598 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 556/1103] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:21:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:50.944714 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6b0be-034c-49b5-94af-fddff630907b None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:50.945139 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6b0be-034c-49b5-94af-fddff630907b None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:50.945247 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6b0be-034c-49b5-94af-fddff630907b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:51.022749 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-45e6b0be-034c-49b5-94af-fddff630907b None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:51.023946 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-45e6b0be-034c-49b5-94af-fddff630907b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:51.134488 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 548/1104] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:21:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:51.285487 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf924b0-9aae-4371-8207-9cf89aad6871 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:51.287722 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-abf924b0-9aae-4371-8207-9cf89aad6871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:51.383950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:51.449931 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf924b0-9aae-4371-8207-9cf89aad6871 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:51.563473 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461140a0-fe11-4f8a-903a-6f6b8dab0817 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:51.564002 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461140a0-fe11-4f8a-903a-6f6b8dab0817 service nova] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:51.564278 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461140a0-fe11-4f8a-903a-6f6b8dab0817 service nova] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:51.564545 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461140a0-fe11-4f8a-903a-6f6b8dab0817 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:51.566140 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-461140a0-fe11-4f8a-903a-6f6b8dab0817 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:51.617824 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:51.664546 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461140a0-fe11-4f8a-903a-6f6b8dab0817 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:51.666161 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461140a0-fe11-4f8a-903a-6f6b8dab0817 service nova] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:51.684669 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 557/1105] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:52.738513 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ca5a9-2679-4bf6-b605-dd8c18edd225 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:52.739140 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ca5a9-2679-4bf6-b605-dd8c18edd225 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:52.739140 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ca5a9-2679-4bf6-b605-dd8c18edd225 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:52.740580 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 549/1106] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:52.746691 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb459ab8-cc94-49cf-a092-e7c663afd434 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:52.747109 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb459ab8-cc94-49cf-a092-e7c663afd434 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:52.747221 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb459ab8-cc94-49cf-a092-e7c663afd434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:52.836682 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-fb459ab8-cc94-49cf-a092-e7c663afd434 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:52.837897 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-fb459ab8-cc94-49cf-a092-e7c663afd434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:52.942023 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 558/1107] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:21:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:53.094054 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ca5a9-2679-4bf6-b605-dd8c18edd225 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:53.096545 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-f62ca5a9-2679-4bf6-b605-dd8c18edd225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:53.158923 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:53.283483 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ca5a9-2679-4bf6-b605-dd8c18edd225 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:53.421133 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add82f7e-3cd9-4834-94b8-1ff36b5e31fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:53.421744 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add82f7e-3cd9-4834-94b8-1ff36b5e31fd service nova] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:53.422020 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add82f7e-3cd9-4834-94b8-1ff36b5e31fd service nova] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:53.422289 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add82f7e-3cd9-4834-94b8-1ff36b5e31fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:53.423996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-add82f7e-3cd9-4834-94b8-1ff36b5e31fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:53.431521 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:53.520964 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add82f7e-3cd9-4834-94b8-1ff36b5e31fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:53.522863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add82f7e-3cd9-4834-94b8-1ff36b5e31fd service nova] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:53.541661 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 550/1108] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:54.577092 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e042c-dfd5-4c87-8022-0a274f4dc543 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:54.577704 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e042c-dfd5-4c87-8022-0a274f4dc543 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:54.577704 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e042c-dfd5-4c87-8022-0a274f4dc543 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:54.579036 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 559/1109] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:54.585178 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85325dee-ecd7-485a-adec-e5ec7ae54412 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:54.585638 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85325dee-ecd7-485a-adec-e5ec7ae54412 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:54.585730 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85325dee-ecd7-485a-adec-e5ec7ae54412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:54.671207 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-85325dee-ecd7-485a-adec-e5ec7ae54412 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:54.672425 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-85325dee-ecd7-485a-adec-e5ec7ae54412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:54.753260 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:54.783648 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 551/1110] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:21:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:54.932460 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e042c-dfd5-4c87-8022-0a274f4dc543 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:54.934768 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-e39e042c-dfd5-4c87-8022-0a274f4dc543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:54.963617 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:55.117122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e042c-dfd5-4c87-8022-0a274f4dc543 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:55.218622 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:55.247783 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d51c0-20e1-4851-b28e-68bf2d0ee463 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:55.248392 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d51c0-20e1-4851-b28e-68bf2d0ee463 service nova] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:55.248674 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d51c0-20e1-4851-b28e-68bf2d0ee463 service nova] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:55.248945 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d51c0-20e1-4851-b28e-68bf2d0ee463 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:55.250550 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-929d51c0-20e1-4851-b28e-68bf2d0ee463 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:55.354049 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929d51c0-20e1-4851-b28e-68bf2d0ee463 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:55.355696 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929d51c0-20e1-4851-b28e-68bf2d0ee463 service nova] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:55.381307 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 560/1111] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:56.424498 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3d3cd-df93-4189-bc34-e13db532c92a None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:56.425109 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3d3cd-df93-4189-bc34-e13db532c92a None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:56.425109 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3d3cd-df93-4189-bc34-e13db532c92a None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:56.426456 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 552/1112] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:21:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:56.432749 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211f823-863d-47ce-91bc-89905cab1471 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:56.433218 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211f823-863d-47ce-91bc-89905cab1471 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:56.433324 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211f823-863d-47ce-91bc-89905cab1471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:56.530741 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-9211f823-863d-47ce-91bc-89905cab1471 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:56.532371 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-9211f823-863d-47ce-91bc-89905cab1471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:56.578477 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:56.638210 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 561/1113] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:21:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:56.792098 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3d3cd-df93-4189-bc34-e13db532c92a None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:56.793546 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-0fe3d3cd-df93-4189-bc34-e13db532c92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:56.958028 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3d3cd-df93-4189-bc34-e13db532c92a None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:57.017211 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:57.079817 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d838f18f-f2dc-4341-b35f-b8771aa45b8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:57.080345 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838f18f-f2dc-4341-b35f-b8771aa45b8d service nova] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:57.080618 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838f18f-f2dc-4341-b35f-b8771aa45b8d service nova] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:57.080886 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838f18f-f2dc-4341-b35f-b8771aa45b8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:57.082477 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-d838f18f-f2dc-4341-b35f-b8771aa45b8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:57.185275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d838f18f-f2dc-4341-b35f-b8771aa45b8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:57.186903 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d838f18f-f2dc-4341-b35f-b8771aa45b8d service nova] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:57.205088 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 553/1114] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:21:58.253394 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcf0bb-e02b-4dc8-a3d0-0c1e676c28cd None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:58.254275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcf0bb-e02b-4dc8-a3d0-0c1e676c28cd None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:58.254275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcf0bb-e02b-4dc8-a3d0-0c1e676c28cd None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:58.256456 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 562/1115] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:21:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:21:58.262985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb99d42-15f6-461f-a64d-7833f0aafebb None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:58.263431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb99d42-15f6-461f-a64d-7833f0aafebb None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:58.263525 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb99d42-15f6-461f-a64d-7833f0aafebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:58.349140 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-ccb99d42-15f6-461f-a64d-7833f0aafebb None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:21:58.350335 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-ccb99d42-15f6-461f-a64d-7833f0aafebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:58.352366 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:58.456998 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 554/1116] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:21:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:21:58.609143 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bcf0bb-e02b-4dc8-a3d0-0c1e676c28cd None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:21:58.610691 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-e2bcf0bb-e02b-4dc8-a3d0-0c1e676c28cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:58.773731 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bcf0bb-e02b-4dc8-a3d0-0c1e676c28cd None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:21:58.803942 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:21:58.898880 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800bb62-1cf4-4f65-9a20-d9f7a1893495 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:21:58.899407 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800bb62-1cf4-4f65-9a20-d9f7a1893495 service nova] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:21:58.899784 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800bb62-1cf4-4f65-9a20-d9f7a1893495 service nova] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:21:58.900230 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800bb62-1cf4-4f65-9a20-d9f7a1893495 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:21:58.901620 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-e800bb62-1cf4-4f65-9a20-d9f7a1893495 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:21:58.994754 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e800bb62-1cf4-4f65-9a20-d9f7a1893495 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:21:58.996413 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e800bb62-1cf4-4f65-9a20-d9f7a1893495 service nova] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:21:59.015607 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 563/1117] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:21:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:01.088470 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0649b6c-d87e-46ac-848e-9cc174e2c66b None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:01.089407 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0649b6c-d87e-46ac-848e-9cc174e2c66b None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:01.089407 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0649b6c-d87e-46ac-848e-9cc174e2c66b None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:01.091553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 555/1118] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:22:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:01.100620 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dccd6d-ac34-4b2a-932c-fd85ab9dcb76 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:01.101090 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dccd6d-ac34-4b2a-932c-fd85ab9dcb76 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:01.101187 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dccd6d-ac34-4b2a-932c-fd85ab9dcb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:01.149157 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:01.177330 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-67dccd6d-ac34-4b2a-932c-fd85ab9dcb76 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:01.178484 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-67dccd6d-ac34-4b2a-932c-fd85ab9dcb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:01.276441 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 564/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:22:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:01.562829 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0649b6c-d87e-46ac-848e-9cc174e2c66b None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:01.564416 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-a0649b6c-d87e-46ac-848e-9cc174e2c66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:01.672957 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0649b6c-d87e-46ac-848e-9cc174e2c66b None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:01.706020 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:01.751956 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f21be0-d60d-48a8-ac1f-f8eed3d94e95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:01.752559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f21be0-d60d-48a8-ac1f-f8eed3d94e95 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:01.752838 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f21be0-d60d-48a8-ac1f-f8eed3d94e95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:01.753111 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f21be0-d60d-48a8-ac1f-f8eed3d94e95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:01.754739 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-54f21be0-d60d-48a8-ac1f-f8eed3d94e95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:01.882136 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f21be0-d60d-48a8-ac1f-f8eed3d94e95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:01.883657 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f21be0-d60d-48a8-ac1f-f8eed3d94e95 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:01.904884 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 556/1120] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:22:04.203739 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb34307-9c86-48ef-9c0c-8e6fe7295893 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:04.204676 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb34307-9c86-48ef-9c0c-8e6fe7295893 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:04.204676 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb34307-9c86-48ef-9c0c-8e6fe7295893 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:04.206698 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 565/1121] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:22:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:04.213093 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb49d6-3df3-4373-b8e7-3c9b9ed35fbe None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:04.213541 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb49d6-3df3-4373-b8e7-3c9b9ed35fbe None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:04.213855 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb49d6-3df3-4373-b8e7-3c9b9ed35fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:04.291542 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-cdcb49d6-3df3-4373-b8e7-3c9b9ed35fbe None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:04.292761 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-cdcb49d6-3df3-4373-b8e7-3c9b9ed35fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:04.306120 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:04.395668 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 557/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:04.684914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb34307-9c86-48ef-9c0c-8e6fe7295893 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:04.686473 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-beb34307-9c86-48ef-9c0c-8e6fe7295893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:04.793619 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb34307-9c86-48ef-9c0c-8e6fe7295893 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:04.807473 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:04.870796 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391bff6-6b8d-4a1f-8f08-2d363887a0d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:04.871397 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391bff6-6b8d-4a1f-8f08-2d363887a0d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:04.871665 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391bff6-6b8d-4a1f-8f08-2d363887a0d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:04.871932 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391bff6-6b8d-4a1f-8f08-2d363887a0d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:04.873500 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-4391bff6-6b8d-4a1f-8f08-2d363887a0d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:04.999191 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4391bff6-6b8d-4a1f-8f08-2d363887a0d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:05.000678 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4391bff6-6b8d-4a1f-8f08-2d363887a0d2 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:05.019883 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 566/1123] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:07.486198 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57cb8ff-9f28-4f45-986a-ce4ea1893196 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:07.487177 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57cb8ff-9f28-4f45-986a-ce4ea1893196 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:07.487177 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57cb8ff-9f28-4f45-986a-ce4ea1893196 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:07.488992 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 558/1124] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:22:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:07.495599 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3b96e-4519-4829-922b-c68c67028415 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:07.496346 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3b96e-4519-4829-922b-c68c67028415 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:07.496509 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3b96e-4519-4829-922b-c68c67028415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:07.552406 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:07.571404 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-84d3b96e-4519-4829-922b-c68c67028415 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:07.572738 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-84d3b96e-4519-4829-922b-c68c67028415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:07.678881 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 567/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:07.968552 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57cb8ff-9f28-4f45-986a-ce4ea1893196 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:07.971074 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-e57cb8ff-9f28-4f45-986a-ce4ea1893196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:08.094392 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57cb8ff-9f28-4f45-986a-ce4ea1893196 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:08.137939 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:08.175049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653fb42-5510-4569-a2c0-ea542ded0ec5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:08.175659 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653fb42-5510-4569-a2c0-ea542ded0ec5 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:08.175932 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653fb42-5510-4569-a2c0-ea542ded0ec5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:08.176199 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653fb42-5510-4569-a2c0-ea542ded0ec5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:08.177777 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-a653fb42-5510-4569-a2c0-ea542ded0ec5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:08.303996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a653fb42-5510-4569-a2c0-ea542ded0ec5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:08.305359 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a653fb42-5510-4569-a2c0-ea542ded0ec5 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:08.325550 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 559/1126] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:10.614120 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67907b10-da4f-4b23-a09b-36494b2b64a2 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:10.614587 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67907b10-da4f-4b23-a09b-36494b2b64a2 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:10.614723 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67907b10-da4f-4b23-a09b-36494b2b64a2 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:10.616078 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 568/1127] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:10.621787 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e41a06-759c-4418-ad75-26c6ef7ab071 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:10.622244 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e41a06-759c-4418-ad75-26c6ef7ab071 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:10.622347 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e41a06-759c-4418-ad75-26c6ef7ab071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:10.694908 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-a3e41a06-759c-4418-ad75-26c6ef7ab071 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:10.696126 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-a3e41a06-759c-4418-ad75-26c6ef7ab071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:10.724096 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:10.804485 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 560/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:22:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:11.088639 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67907b10-da4f-4b23-a09b-36494b2b64a2 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:11.090995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-67907b10-da4f-4b23-a09b-36494b2b64a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:11.211554 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67907b10-da4f-4b23-a09b-36494b2b64a2 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:11.236341 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:11.299488 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a0ba41-8b7a-462d-8ab3-07aafa1ee666 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:11.300090 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a0ba41-8b7a-462d-8ab3-07aafa1ee666 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:11.300364 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a0ba41-8b7a-462d-8ab3-07aafa1ee666 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:11.300630 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a0ba41-8b7a-462d-8ab3-07aafa1ee666 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:11.302216 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-f1a0ba41-8b7a-462d-8ab3-07aafa1ee666 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:11.417176 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a0ba41-8b7a-462d-8ab3-07aafa1ee666 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:11.418627 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a0ba41-8b7a-462d-8ab3-07aafa1ee666 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:11.438973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 569/1129] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:13.720444 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5e23e-57f0-4749-a661-78a75a67bbe4 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:13.721315 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5e23e-57f0-4749-a661-78a75a67bbe4 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:13.721315 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5e23e-57f0-4749-a661-78a75a67bbe4 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:13.723535 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 561/1130] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:22:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:13.732913 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335820b-abc3-4452-acc4-0d7cd3d90d47 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:13.733615 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335820b-abc3-4452-acc4-0d7cd3d90d47 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:13.733788 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335820b-abc3-4452-acc4-0d7cd3d90d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:13.802807 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:13.806092 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-5335820b-abc3-4452-acc4-0d7cd3d90d47 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:13.807961 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-5335820b-abc3-4452-acc4-0d7cd3d90d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:13.909856 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 570/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:14.196778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5e23e-57f0-4749-a661-78a75a67bbe4 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:14.199036 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-ada5e23e-57f0-4749-a661-78a75a67bbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:14.313060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5e23e-57f0-4749-a661-78a75a67bbe4 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:14.365024 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:14.390640 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ce18d-7327-422b-9a28-dcc064186839 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:14.391342 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ce18d-7327-422b-9a28-dcc064186839 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:14.391612 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ce18d-7327-422b-9a28-dcc064186839 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:14.391876 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ce18d-7327-422b-9a28-dcc064186839 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:14.393441 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-9c1ce18d-7327-422b-9a28-dcc064186839 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:14.518861 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ce18d-7327-422b-9a28-dcc064186839 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:14.520308 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ce18d-7327-422b-9a28-dcc064186839 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:14.539010 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 562/1132] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:16.763935 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b91cbd-7a63-4b65-9ee1-a0e466ab600c None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:16.764491 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b91cbd-7a63-4b65-9ee1-a0e466ab600c None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:16.764491 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b91cbd-7a63-4b65-9ee1-a0e466ab600c None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:16.766579 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 571/1133] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:22:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:16.772399 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d3022-c8fd-4a44-9e2d-5ef97be2011f None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:16.772857 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d3022-c8fd-4a44-9e2d-5ef97be2011f None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:16.772970 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d3022-c8fd-4a44-9e2d-5ef97be2011f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:16.851844 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-c01d3022-c8fd-4a44-9e2d-5ef97be2011f None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:16.853057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-c01d3022-c8fd-4a44-9e2d-5ef97be2011f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:16.882608 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:16.952604 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 563/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:17.248193 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b91cbd-7a63-4b65-9ee1-a0e466ab600c None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:17.250644 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-c9b91cbd-7a63-4b65-9ee1-a0e466ab600c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:17.374843 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b91cbd-7a63-4b65-9ee1-a0e466ab600c None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:17.407990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:17.457212 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bec863e-e792-458b-b3ac-51d72e42f479 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:17.457822 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec863e-e792-458b-b3ac-51d72e42f479 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:17.458095 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec863e-e792-458b-b3ac-51d72e42f479 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:17.458361 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec863e-e792-458b-b3ac-51d72e42f479 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:17.459989 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-2bec863e-e792-458b-b3ac-51d72e42f479 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:17.585283 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bec863e-e792-458b-b3ac-51d72e42f479 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:17.586630 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bec863e-e792-458b-b3ac-51d72e42f479 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:17.612080 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 572/1135] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:19.836861 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6c862-999f-45da-93ff-6248fdc89712 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:19.837463 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6c862-999f-45da-93ff-6248fdc89712 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:19.837463 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6c862-999f-45da-93ff-6248fdc89712 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:19.839632 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 564/1136] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:22:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:19.846272 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e563a-215d-47c4-a2fd-c878c14e5bf5 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:19.846719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e563a-215d-47c4-a2fd-c878c14e5bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:19.846861 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e563a-215d-47c4-a2fd-c878c14e5bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:19.924670 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-4f1e563a-215d-47c4-a2fd-c878c14e5bf5 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:19.925806 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-4f1e563a-215d-47c4-a2fd-c878c14e5bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:19.934835 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:20.029812 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 573/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:20.315436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6c862-999f-45da-93ff-6248fdc89712 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:20.317850 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-96b6c862-999f-45da-93ff-6248fdc89712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:20.438181 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6c862-999f-45da-93ff-6248fdc89712 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:20.497259 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:20.516720 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854cd7d-def3-4377-9900-bc140b142b3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:20.517336 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854cd7d-def3-4377-9900-bc140b142b3a service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:20.517595 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854cd7d-def3-4377-9900-bc140b142b3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:20.517852 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854cd7d-def3-4377-9900-bc140b142b3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:20.519481 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-6854cd7d-def3-4377-9900-bc140b142b3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:20.643523 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6854cd7d-def3-4377-9900-bc140b142b3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:20.644977 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6854cd7d-def3-4377-9900-bc140b142b3a service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:20.663575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 565/1138] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:22.872655 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c5e8c-58f8-4235-bfd3-d9dadc09d43a None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:22.873486 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c5e8c-58f8-4235-bfd3-d9dadc09d43a None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:22.873486 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c5e8c-58f8-4235-bfd3-d9dadc09d43a None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:22.874423 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 574/1139] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:22.880182 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062e781-6b90-445a-b939-0db6996ebd2b None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:22.880642 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062e781-6b90-445a-b939-0db6996ebd2b None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:22.880772 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062e781-6b90-445a-b939-0db6996ebd2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:22.950700 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-5062e781-6b90-445a-b939-0db6996ebd2b None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:22.951956 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-5062e781-6b90-445a-b939-0db6996ebd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:22.992965 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:23.057435 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 566/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:22:23.338252 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c5e8c-58f8-4235-bfd3-d9dadc09d43a None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:23.339868 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-ae9c5e8c-58f8-4235-bfd3-d9dadc09d43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:23.449727 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c5e8c-58f8-4235-bfd3-d9dadc09d43a None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:23.498053 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:23.535721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b24c5c-44c1-4885-8574-481c6a8bf982 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:23.536323 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b24c5c-44c1-4885-8574-481c6a8bf982 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:23.536597 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b24c5c-44c1-4885-8574-481c6a8bf982 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:23.536868 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b24c5c-44c1-4885-8574-481c6a8bf982 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:23.538448 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-26b24c5c-44c1-4885-8574-481c6a8bf982 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:23.660456 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b24c5c-44c1-4885-8574-481c6a8bf982 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:23.661911 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b24c5c-44c1-4885-8574-481c6a8bf982 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:23.680570 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 575/1141] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:25.828106 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f9eee-8d1d-4a52-9124-e70cef72ded0 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:25.829050 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f9eee-8d1d-4a52-9124-e70cef72ded0 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:25.829050 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f9eee-8d1d-4a52-9124-e70cef72ded0 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:25.831230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 567/1142] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:22:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:25.840255 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae4ada-ed41-4d58-b209-fa0bba799cac None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:25.841017 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae4ada-ed41-4d58-b209-fa0bba799cac None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:25.841312 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae4ada-ed41-4d58-b209-fa0bba799cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:25.919138 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-b9ae4ada-ed41-4d58-b209-fa0bba799cac None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:25.920279 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-b9ae4ada-ed41-4d58-b209-fa0bba799cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:25.939907 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:26.035347 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 576/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:26.320357 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f9eee-8d1d-4a52-9124-e70cef72ded0 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:26.321933 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-158f9eee-8d1d-4a52-9124-e70cef72ded0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:26.430370 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f9eee-8d1d-4a52-9124-e70cef72ded0 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:26.499339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:26.508746 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca69e53e-9349-41af-940e-e5498a5cc6ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:26.509659 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69e53e-9349-41af-940e-e5498a5cc6ed service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:26.510059 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69e53e-9349-41af-940e-e5498a5cc6ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:26.510315 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69e53e-9349-41af-940e-e5498a5cc6ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:26.512860 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-ca69e53e-9349-41af-940e-e5498a5cc6ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:26.637277 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca69e53e-9349-41af-940e-e5498a5cc6ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:26.638912 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca69e53e-9349-41af-940e-e5498a5cc6ed service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:26.658008 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 568/1144] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:28.829248 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428faa5c-5ec1-4b88-87ce-b780c3b9857e None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:28.830065 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428faa5c-5ec1-4b88-87ce-b780c3b9857e None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:28.830065 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428faa5c-5ec1-4b88-87ce-b780c3b9857e None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:28.832306 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 577/1145] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:22:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:28.841366 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d97d2-e5fb-4328-8f32-a7116e5e3638 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:28.841795 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d97d2-e5fb-4328-8f32-a7116e5e3638 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:28.842084 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d97d2-e5fb-4328-8f32-a7116e5e3638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:28.916273 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-cf0d97d2-e5fb-4328-8f32-a7116e5e3638 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:28.917524 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-cf0d97d2-e5fb-4328-8f32-a7116e5e3638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:28.971591 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:29.025229 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 569/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:22:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:29.311194 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428faa5c-5ec1-4b88-87ce-b780c3b9857e None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:29.313540 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-428faa5c-5ec1-4b88-87ce-b780c3b9857e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:29.422238 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428faa5c-5ec1-4b88-87ce-b780c3b9857e None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:29.477058 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:29.504573 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d722c678-425d-4b5e-a061-cd2723fd3948 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:29.505176 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722c678-425d-4b5e-a061-cd2723fd3948 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:29.505445 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722c678-425d-4b5e-a061-cd2723fd3948 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:29.505712 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722c678-425d-4b5e-a061-cd2723fd3948 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:29.507320 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-d722c678-425d-4b5e-a061-cd2723fd3948 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:29.619560 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d722c678-425d-4b5e-a061-cd2723fd3948 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:29.621307 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d722c678-425d-4b5e-a061-cd2723fd3948 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:29.640024 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 578/1147] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:39.023719 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=22 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:39.637321 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=66 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:49.658350 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af360f-dc72-4caa-8db4-7b0a475c5cc2 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:49.659144 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af360f-dc72-4caa-8db4-7b0a475c5cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:49.659144 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af360f-dc72-4caa-8db4-7b0a475c5cc2 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:49.661009 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 570/1148] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:22:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:49.666904 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd199a-5d38-4b71-bf03-d49af42a442b None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:49.667342 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd199a-5d38-4b71-bf03-d49af42a442b None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:49.667438 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd199a-5d38-4b71-bf03-d49af42a442b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:49.735430 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-18cd199a-5d38-4b71-bf03-d49af42a442b None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:49.736638 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-18cd199a-5d38-4b71-bf03-d49af42a442b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:49.846526 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 579/1149] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:50.141268 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f512b57-687c-40b6-be35-33c8b0c9963a None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:50.141550 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f512b57-687c-40b6-be35-33c8b0c9963a None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:50.141818 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f512b57-687c-40b6-be35-33c8b0c9963a None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:50.143238 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 571/1150] 10.222.0.30 () {54 vars in 962 bytes} [Wed Nov 2 07:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:50.148376 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d5daa-01f0-4c47-951e-66070539f615 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:50.148656 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d5daa-01f0-4c47-951e-66070539f615 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:50.148921 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d5daa-01f0-4c47-951e-66070539f615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:50.229673 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-c09d5daa-01f0-4c47-951e-66070539f615 None None] MFA Rules not processed for user `9a051ca8154540ad93db6d028b8323a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:50.230910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-c09d5daa-01f0-4c47-951e-66070539f615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:50.253049 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:50.312400 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 580/1151] 10.222.0.30 () {58 vars in 1059 bytes} [Wed Nov 2 07:22:50 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:50.320453 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f512b57-687c-40b6-be35-33c8b0c9963a None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:50.321984 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-8f512b57-687c-40b6-be35-33c8b0c9963a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:50.441244 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f512b57-687c-40b6-be35-33c8b0c9963a None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:50.517612 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9d60a-9fd1-4b7d-9723-04f364ec7402 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a051ca8154540ad93db6d028b8323a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:50.518211 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9d60a-9fd1-4b7d-9723-04f364ec7402 service glance] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:50.518478 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9d60a-9fd1-4b7d-9723-04f364ec7402 service glance] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:50.518753 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9d60a-9fd1-4b7d-9723-04f364ec7402 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:50.520328 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-f7a9d60a-9fd1-4b7d-9723-04f364ec7402 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:50.576686 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:50.634361 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a9d60a-9fd1-4b7d-9723-04f364ec7402 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:50.635841 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a9d60a-9fd1-4b7d-9723-04f364ec7402 service glance] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:50.654379 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 572/1152] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:52.886698 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7abb5-ce47-426c-bbfa-9660ea3891bc None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:52.887301 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7abb5-ce47-426c-bbfa-9660ea3891bc None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:52.887301 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7abb5-ce47-426c-bbfa-9660ea3891bc None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:52.888607 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 581/1153] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:52.894881 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3040d28-57ea-48c7-b781-ea6764c1bc60 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:52.895340 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3040d28-57ea-48c7-b781-ea6764c1bc60 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:52.895450 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3040d28-57ea-48c7-b781-ea6764c1bc60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:52.968421 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-e3040d28-57ea-48c7-b781-ea6764c1bc60 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:52.969675 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-e3040d28-57ea-48c7-b781-ea6764c1bc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:53.070707 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 573/1154] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:22:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:53.348185 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a7abb5-ce47-426c-bbfa-9660ea3891bc None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:53.350537 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-50a7abb5-ce47-426c-bbfa-9660ea3891bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:53.497817 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a7abb5-ce47-426c-bbfa-9660ea3891bc None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:53.548943 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:53.594059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e2839-e1b8-4199-8766-a71cd42449c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:53.594698 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e2839-e1b8-4199-8766-a71cd42449c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:53.595059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e2839-e1b8-4199-8766-a71cd42449c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:53.595197 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e2839-e1b8-4199-8766-a71cd42449c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:53.595864 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e2839-e1b8-4199-8766-a71cd42449c3 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:53.596892 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: WARNING py.warnings [None req-b12e2839-e1b8-4199-8766-a71cd42449c3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:22:53.596892 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: warnings.warn(msg) Nov 02 07:22:53.596892 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: Nov 02 07:22:53.598025 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e2839-e1b8-4199-8766-a71cd42449c3 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:53.618017 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 582/1155] 10.222.0.30 () {56 vars in 1108 bytes} [Wed Nov 2 07:22:53 2022] GET /identity/v3/projects => generated 2553 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:22:55.785656 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2b8cf-7944-4cd0-9836-d8b631af1a87 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:55.786556 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2b8cf-7944-4cd0-9836-d8b631af1a87 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:55.786556 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2b8cf-7944-4cd0-9836-d8b631af1a87 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:55.789188 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 574/1156] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:22:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:55.795661 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead80cb0-2056-4582-ae96-1ad26215cc2d None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:55.796112 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead80cb0-2056-4582-ae96-1ad26215cc2d None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:55.796222 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead80cb0-2056-4582-ae96-1ad26215cc2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:55.880998 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-ead80cb0-2056-4582-ae96-1ad26215cc2d None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:55.882120 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-ead80cb0-2056-4582-ae96-1ad26215cc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:55.993378 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 583/1157] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:22:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:56.285978 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2b8cf-7944-4cd0-9836-d8b631af1a87 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:56.288122 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-13d2b8cf-7944-4cd0-9836-d8b631af1a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:56.301294 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:56.449650 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2b8cf-7944-4cd0-9836-d8b631af1a87 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:56.542148 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:56.571114 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea784b-a18e-4d6e-81c7-71fed84038ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:56.571712 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea784b-a18e-4d6e-81c7-71fed84038ab service nova] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:56.571977 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea784b-a18e-4d6e-81c7-71fed84038ab service nova] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:56.572233 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea784b-a18e-4d6e-81c7-71fed84038ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:56.573804 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-45ea784b-a18e-4d6e-81c7-71fed84038ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:56.665034 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea784b-a18e-4d6e-81c7-71fed84038ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:56.666650 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea784b-a18e-4d6e-81c7-71fed84038ab service nova] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:56.685606 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 575/1158] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:22:59.026943 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cd968-54a6-4dab-8c81-381bd134c107 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:59.027885 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cd968-54a6-4dab-8c81-381bd134c107 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:59.027885 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cd968-54a6-4dab-8c81-381bd134c107 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:59.030039 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 584/1159] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:22:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:22:59.036577 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a665b-3b5d-4e72-ba68-63a9de4bbdf4 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:59.037625 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a665b-3b5d-4e72-ba68-63a9de4bbdf4 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:59.037773 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a665b-3b5d-4e72-ba68-63a9de4bbdf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:59.114072 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-0d4a665b-3b5d-4e72-ba68-63a9de4bbdf4 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:22:59.115326 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-0d4a665b-3b5d-4e72-ba68-63a9de4bbdf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:59.161180 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:59.216090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 576/1160] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:22:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:22:59.507348 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245cd968-54a6-4dab-8c81-381bd134c107 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:22:59.509964 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-245cd968-54a6-4dab-8c81-381bd134c107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:59.544304 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:59.627859 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245cd968-54a6-4dab-8c81-381bd134c107 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:22:59.703153 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd96df-1961-41f9-a1f5-23fcc83f74d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:22:59.703823 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd96df-1961-41f9-a1f5-23fcc83f74d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:22:59.704152 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd96df-1961-41f9-a1f5-23fcc83f74d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:22:59.704475 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd96df-1961-41f9-a1f5-23fcc83f74d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:22:59.706164 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-6ddd96df-1961-41f9-a1f5-23fcc83f74d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:22:59.779821 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:22:59.815707 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddd96df-1961-41f9-a1f5-23fcc83f74d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:22:59.817117 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddd96df-1961-41f9-a1f5-23fcc83f74d3 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:22:59.835878 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 585/1161] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:22:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:23:02.055923 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40c0f1-7774-4aed-844c-50f44131ceec None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:02.056628 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40c0f1-7774-4aed-844c-50f44131ceec None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:02.056628 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40c0f1-7774-4aed-844c-50f44131ceec None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:02.057724 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 577/1162] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:23:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:23:02.063493 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6bdd4d-364e-487e-9cba-4d65252fb417 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:02.064287 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6bdd4d-364e-487e-9cba-4d65252fb417 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:02.064287 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6bdd4d-364e-487e-9cba-4d65252fb417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:02.145610 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-5e6bdd4d-364e-487e-9cba-4d65252fb417 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:23:02.146837 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-5e6bdd4d-364e-487e-9cba-4d65252fb417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:02.251743 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 586/1163] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:23:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:23:02.544113 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c40c0f1-7774-4aed-844c-50f44131ceec None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:23:02.546486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-0c40c0f1-7774-4aed-844c-50f44131ceec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:02.663663 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c40c0f1-7774-4aed-844c-50f44131ceec None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:23:02.741406 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:02.743028 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2aa383-20ee-4d45-8bec-af797856fdbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:23:02.743727 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2aa383-20ee-4d45-8bec-af797856fdbd service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:02.744020 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2aa383-20ee-4d45-8bec-af797856fdbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:02.744313 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2aa383-20ee-4d45-8bec-af797856fdbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:02.746402 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-2b2aa383-20ee-4d45-8bec-af797856fdbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:02.864405 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2aa383-20ee-4d45-8bec-af797856fdbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:23:02.865755 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2aa383-20ee-4d45-8bec-af797856fdbd service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:23:02.884303 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 578/1164] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:23:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:23:05.142053 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb3a98-8461-44db-b116-cb9d7ca09f0b None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:05.142681 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb3a98-8461-44db-b116-cb9d7ca09f0b None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:05.142681 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb3a98-8461-44db-b116-cb9d7ca09f0b None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:05.144065 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 587/1165] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:23:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:23:05.150638 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83244101-0107-4ae4-be91-86917e53000a None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:05.151157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83244101-0107-4ae4-be91-86917e53000a None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:05.151157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83244101-0107-4ae4-be91-86917e53000a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:05.225856 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-83244101-0107-4ae4-be91-86917e53000a None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:23:05.227119 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-83244101-0107-4ae4-be91-86917e53000a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:05.282157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:05.332215 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 579/1166] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:23:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:23:05.626393 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb3a98-8461-44db-b116-cb9d7ca09f0b None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:23:05.627988 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-d6bb3a98-8461-44db-b116-cb9d7ca09f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:05.666355 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:05.737488 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb3a98-8461-44db-b116-cb9d7ca09f0b None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:23:05.822850 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af66f0c-e490-4fde-a6d7-22df207d7268 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:23:05.823458 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af66f0c-e490-4fde-a6d7-22df207d7268 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:05.823734 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af66f0c-e490-4fde-a6d7-22df207d7268 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:05.824000 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af66f0c-e490-4fde-a6d7-22df207d7268 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:05.825591 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-6af66f0c-e490-4fde-a6d7-22df207d7268 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:05.909758 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:05.941290 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af66f0c-e490-4fde-a6d7-22df207d7268 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:23:05.942769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af66f0c-e490-4fde-a6d7-22df207d7268 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:23:05.962117 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 588/1167] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:23:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:23:08.305851 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830f5b0-b957-4c44-8df4-d72c44e578cf None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:08.306808 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830f5b0-b957-4c44-8df4-d72c44e578cf None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:08.306808 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830f5b0-b957-4c44-8df4-d72c44e578cf None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:08.308950 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 580/1168] 10.222.0.30 () {54 vars in 927 bytes} [Wed Nov 2 07:23:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:23:08.314508 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787dd86-96fa-48f1-80ce-26a2813a4b37 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:08.315007 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787dd86-96fa-48f1-80ce-26a2813a4b37 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:08.315196 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787dd86-96fa-48f1-80ce-26a2813a4b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:08.382387 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-d787dd86-96fa-48f1-80ce-26a2813a4b37 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:23:08.383623 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-d787dd86-96fa-48f1-80ce-26a2813a4b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:08.482893 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 589/1169] 10.222.0.30 () {58 vars in 1024 bytes} [Wed Nov 2 07:23:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:23:08.769275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830f5b0-b957-4c44-8df4-d72c44e578cf None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:23:08.771807 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-f830f5b0-b957-4c44-8df4-d72c44e578cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:08.895936 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830f5b0-b957-4c44-8df4-d72c44e578cf None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:23:08.971755 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbfd619-2841-45ee-9118-ab94159e2e61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:23:08.972363 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbfd619-2841-45ee-9118-ab94159e2e61 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:23:08.972640 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbfd619-2841-45ee-9118-ab94159e2e61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:23:08.972876 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbfd619-2841-45ee-9118-ab94159e2e61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:23:08.974504 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-ccbfd619-2841-45ee-9118-ab94159e2e61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:23:08.983710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:09.089808 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbfd619-2841-45ee-9118-ab94159e2e61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:23:09.091287 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbfd619-2841-45ee-9118-ab94159e2e61 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:23:09.109755 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 581/1170] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:23:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:23:18.481308 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=82 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:23:19.107215 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=50 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:23.802700 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e815b-a3d4-4416-a9d9-7ba0e762c57f None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:23.803642 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e815b-a3d4-4416-a9d9-7ba0e762c57f None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:23.803642 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e815b-a3d4-4416-a9d9-7ba0e762c57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:23.882756 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-411e815b-a3d4-4416-a9d9-7ba0e762c57f None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:24:23.884168 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-411e815b-a3d4-4416-a9d9-7ba0e762c57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:23.990424 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 590/1171] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:24:23.998751 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a9155-7588-44e3-92bb-3471372e1850 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:23.999275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a9155-7588-44e3-92bb-3471372e1850 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:23.999275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a9155-7588-44e3-92bb-3471372e1850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:24.087387 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-105a9155-7588-44e3-92bb-3471372e1850 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:24:24.088913 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-105a9155-7588-44e3-92bb-3471372e1850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:24.201976 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 582/1172] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:24:24.206917 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e815b-a3d4-4416-a9d9-7ba0e762c57f None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:24:24.208124 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-411e815b-a3d4-4416-a9d9-7ba0e762c57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:24.220294 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e815b-a3d4-4416-a9d9-7ba0e762c57f None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:24:24.295721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:24.331890 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290f3f8-a354-4804-bbaa-f23a910d8feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:24:24.332523 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290f3f8-a354-4804-bbaa-f23a910d8feb admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:24.332794 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290f3f8-a354-4804-bbaa-f23a910d8feb admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:24.333081 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290f3f8-a354-4804-bbaa-f23a910d8feb admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:24.333722 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6290f3f8-a354-4804-bbaa-f23a910d8feb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:24:24.334014 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6290f3f8-a354-4804-bbaa-f23a910d8feb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:24:24.335068 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: WARNING py.warnings [None req-6290f3f8-a354-4804-bbaa-f23a910d8feb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:24:24.335068 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: warnings.warn(msg) Nov 02 07:24:24.335068 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: Nov 02 07:24:24.335937 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6290f3f8-a354-4804-bbaa-f23a910d8feb admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:24:24.348881 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 591/1173] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:24:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:24:24.360259 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a9155-7588-44e3-92bb-3471372e1850 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:24:24.369833 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a9155-7588-44e3-92bb-3471372e1850 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:24:24.438042 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:24.474727 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9ca41-790b-401b-98ad-e8fcd1effef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:24:24.475274 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9ca41-790b-401b-98ad-e8fcd1effef1 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:24.475546 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9ca41-790b-401b-98ad-e8fcd1effef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:24.475808 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9ca41-790b-401b-98ad-e8fcd1effef1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:24.476431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b9ca41-790b-401b-98ad-e8fcd1effef1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:24:24.477545 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: WARNING py.warnings [None req-f8b9ca41-790b-401b-98ad-e8fcd1effef1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:24:24.477545 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: warnings.warn(msg) Nov 02 07:24:24.477545 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: Nov 02 07:24:24.478757 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b9ca41-790b-401b-98ad-e8fcd1effef1 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:24:24.540986 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 583/1174] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:24:24 2022] POST /identity/v3/projects => generated 377 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:24:24.545833 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290f3f8-a354-4804-bbaa-f23a910d8feb admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:24:24.547400 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-6290f3f8-a354-4804-bbaa-f23a910d8feb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:24.580377 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290f3f8-a354-4804-bbaa-f23a910d8feb admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:24:24.680107 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c9dabd-7e22-4f9b-a767-330ba928525b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:24:24.680717 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9dabd-7e22-4f9b-a767-330ba928525b admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:24.681107 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9dabd-7e22-4f9b-a767-330ba928525b admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:24.681471 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9dabd-7e22-4f9b-a767-330ba928525b admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:24.682122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c9dabd-7e22-4f9b-a767-330ba928525b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:24:24.683205 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: WARNING py.warnings [None req-78c9dabd-7e22-4f9b-a767-330ba928525b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:24:24.683205 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: warnings.warn(msg) Nov 02 07:24:24.683205 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: Nov 02 07:24:24.684426 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c9dabd-7e22-4f9b-a767-330ba928525b admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:24:24.730702 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 592/1175] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:24:24 2022] POST /identity/v3/users => generated 289 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:24:24.735034 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9ca41-790b-401b-98ad-e8fcd1effef1 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:24:24.736551 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-f8b9ca41-790b-401b-98ad-e8fcd1effef1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:24.802558 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9ca41-790b-401b-98ad-e8fcd1effef1 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:24:24.895165 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:24.907399 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df0acc-0f10-47a4-8225-66b95f389947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:24:24.908011 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df0acc-0f10-47a4-8225-66b95f389947 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:24.908286 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df0acc-0f10-47a4-8225-66b95f389947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:24.908692 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df0acc-0f10-47a4-8225-66b95f389947 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:24.909335 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df0acc-0f10-47a4-8225-66b95f389947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:24:24.910992 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df0acc-0f10-47a4-8225-66b95f389947 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:24:24.925151 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 584/1176] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:24:24.929580 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c9dabd-7e22-4f9b-a767-330ba928525b admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:24:24.931278 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-78c9dabd-7e22-4f9b-a767-330ba928525b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:24.943230 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c9dabd-7e22-4f9b-a767-330ba928525b admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:24:24.969145 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:25.047290 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6422b6a-585e-4863-9c7b-96efbb3484bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:24:25.047814 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6422b6a-585e-4863-9c7b-96efbb3484bb admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:25.048086 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6422b6a-585e-4863-9c7b-96efbb3484bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:25.048348 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6422b6a-585e-4863-9c7b-96efbb3484bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:25.048979 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6422b6a-585e-4863-9c7b-96efbb3484bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:24:25.050574 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6422b6a-585e-4863-9c7b-96efbb3484bb admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:24:25.064666 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 593/1177] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:24:25.069077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df0acc-0f10-47a4-8225-66b95f389947 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:24:25.070574 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-56df0acc-0f10-47a4-8225-66b95f389947 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:25.105065 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df0acc-0f10-47a4-8225-66b95f389947 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:24:25.206324 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e4510-7eae-4ece-8931-0a6d2aa76373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:24:25.207323 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e4510-7eae-4ece-8931-0a6d2aa76373 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:25.207681 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e4510-7eae-4ece-8931-0a6d2aa76373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:25.208019 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e4510-7eae-4ece-8931-0a6d2aa76373 admin admin] PATH_INFO: `/v3/projects/4dd464b5ec0947dea32aff6e3dd9f6f7/users/54ae21d35f6b461882ec463289891c68/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:25.234225 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39e4510-7eae-4ece-8931-0a6d2aa76373 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dd464b5ec0947dea32aff6e3dd9f6f7, user_id=54ae21d35f6b461882ec463289891c68, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:24:25.236927 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39e4510-7eae-4ece-8931-0a6d2aa76373 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:24:25.264797 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 585/1178] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:24:25 2022] PUT /identity/v3/projects/4dd464b5ec0947dea32aff6e3dd9f6f7/users/54ae21d35f6b461882ec463289891c68/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:24:25.276247 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a04f0-bba4-4ff6-9718-37cf751d21cd None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:25.276534 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a04f0-bba4-4ff6-9718-37cf751d21cd None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:25.276797 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a04f0-bba4-4ff6-9718-37cf751d21cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:25.335360 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-4d9a04f0-bba4-4ff6-9718-37cf751d21cd None None] MFA Rules not processed for user `54ae21d35f6b461882ec463289891c68`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:24:25.336450 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-4d9a04f0-bba4-4ff6-9718-37cf751d21cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:25.415888 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 594/1179] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:24:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:24:25.429601 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96a5a0-3621-4dff-a22d-545812dc32e1 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:25.429883 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96a5a0-3621-4dff-a22d-545812dc32e1 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:25.430153 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96a5a0-3621-4dff-a22d-545812dc32e1 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:25.431605 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 586/1180] 10.222.0.30 () {54 vars in 963 bytes} [Wed Nov 2 07:24:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:24:25.436500 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10475588-366b-408d-8a29-38b26652d76f None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:25.436912 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10475588-366b-408d-8a29-38b26652d76f None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:25.437225 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10475588-366b-408d-8a29-38b26652d76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:25.470451 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:25.515364 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-10475588-366b-408d-8a29-38b26652d76f None None] MFA Rules not processed for user `315861510b1041dca19d1e8753d9b21d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:24:25.516571 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-10475588-366b-408d-8a29-38b26652d76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:25.601962 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 595/1181] 10.222.0.30 () {58 vars in 1060 bytes} [Wed Nov 2 07:24:25 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:24:25.609976 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96a5a0-3621-4dff-a22d-545812dc32e1 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:24:25.611562 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-7f96a5a0-3621-4dff-a22d-545812dc32e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:25.668822 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:25.725679 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96a5a0-3621-4dff-a22d-545812dc32e1 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:24:25.814232 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76970698-d155-404e-83e3-49e253da2b72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:24:25.814866 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76970698-d155-404e-83e3-49e253da2b72 service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:25.815135 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76970698-d155-404e-83e3-49e253da2b72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:25.815396 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76970698-d155-404e-83e3-49e253da2b72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:25.816986 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-76970698-d155-404e-83e3-49e253da2b72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:25.920921 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76970698-d155-404e-83e3-49e253da2b72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:24:25.922294 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76970698-d155-404e-83e3-49e253da2b72 service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:24:25.925859 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:25.942851 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 587/1182] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:24:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:24:25.989664 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0395831-f57b-4cb4-a0f4-2bcb282f23a2 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:25.990153 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0395831-f57b-4cb4-a0f4-2bcb282f23a2 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:25.990153 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0395831-f57b-4cb4-a0f4-2bcb282f23a2 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:25.991495 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 596/1183] 10.222.0.30 () {52 vars in 830 bytes} [Wed Nov 2 07:24:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:24:25.995553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76970698-d155-404e-83e3-49e253da2b72 service cinder] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:24:25.996779 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-76970698-d155-404e-83e3-49e253da2b72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:26.107615 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76970698-d155-404e-83e3-49e253da2b72 service cinder] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:24:26.208827 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422aafc7-4d47-4f45-9f14-fdc2d98a61c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:24:26.209375 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422aafc7-4d47-4f45-9f14-fdc2d98a61c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:26.209648 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422aafc7-4d47-4f45-9f14-fdc2d98a61c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:26.209911 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422aafc7-4d47-4f45-9f14-fdc2d98a61c6 admin admin] PATH_INFO: `/v3/users/54ae21d35f6b461882ec463289891c68` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:26.223519 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422aafc7-4d47-4f45-9f14-fdc2d98a61c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54ae21d35f6b461882ec463289891c68)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:24:26.224556 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: WARNING py.warnings [None req-422aafc7-4d47-4f45-9f14-fdc2d98a61c6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:24:26.224556 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: warnings.warn(msg) Nov 02 07:24:26.224556 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: Nov 02 07:24:26.225778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422aafc7-4d47-4f45-9f14-fdc2d98a61c6 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:24:26.236263 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:26.307706 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-422aafc7-4d47-4f45-9f14-fdc2d98a61c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54ae21d35f6b461882ec463289891c68'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:24:26.308140 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-422aafc7-4d47-4f45-9f14-fdc2d98a61c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54ae21d35f6b461882ec463289891c68'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:24:26.322801 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-422aafc7-4d47-4f45-9f14-fdc2d98a61c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54ae21d35f6b461882ec463289891c68'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:24:26.347974 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-422aafc7-4d47-4f45-9f14-fdc2d98a61c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54ae21d35f6b461882ec463289891c68'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:24:26.359431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-422aafc7-4d47-4f45-9f14-fdc2d98a61c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54ae21d35f6b461882ec463289891c68'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:24:26.365616 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 588/1184] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:24:25 2022] DELETE /identity/v3/users/54ae21d35f6b461882ec463289891c68 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:24:26.370140 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6c242-dfae-44cb-9225-82c04079a47a None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:26.370416 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6c242-dfae-44cb-9225-82c04079a47a None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:26.370706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6c242-dfae-44cb-9225-82c04079a47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:26.437310 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-72f6c242-dfae-44cb-9225-82c04079a47a None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:24:26.439316 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-72f6c242-dfae-44cb-9225-82c04079a47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:26.525303 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:26.538049 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 597/1185] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:24:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:24:26.551310 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422aafc7-4d47-4f45-9f14-fdc2d98a61c6 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:24:26.552566 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-422aafc7-4d47-4f45-9f14-fdc2d98a61c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:26.664633 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422aafc7-4d47-4f45-9f14-fdc2d98a61c6 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:24:26.728360 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:26.754512 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea5600-7892-4412-ae51-9bfceac6ed9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:24:26.755161 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea5600-7892-4412-ae51-9bfceac6ed9e service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:26.755437 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea5600-7892-4412-ae51-9bfceac6ed9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:26.755706 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea5600-7892-4412-ae51-9bfceac6ed9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:26.757320 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-f2ea5600-7892-4412-ae51-9bfceac6ed9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:26.879031 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ea5600-7892-4412-ae51-9bfceac6ed9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:24:26.880484 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ea5600-7892-4412-ae51-9bfceac6ed9e service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:24:26.899494 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 589/1186] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:24:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:24:27.257000 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f6c242-dfae-44cb-9225-82c04079a47a None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:24:27.259189 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-72f6c242-dfae-44cb-9225-82c04079a47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:24:27.271242 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f6c242-dfae-44cb-9225-82c04079a47a None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:24:27.380878 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62cc23-e305-4982-84fe-2ee955cbf774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:24:27.381520 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62cc23-e305-4982-84fe-2ee955cbf774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:24:27.381795 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62cc23-e305-4982-84fe-2ee955cbf774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:24:27.382067 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62cc23-e305-4982-84fe-2ee955cbf774 admin admin] PATH_INFO: `/v3/projects/4dd464b5ec0947dea32aff6e3dd9f6f7` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:24:27.394543 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a62cc23-e305-4982-84fe-2ee955cbf774 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dd464b5ec0947dea32aff6e3dd9f6f7)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:24:27.395573 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: WARNING py.warnings [None req-6a62cc23-e305-4982-84fe-2ee955cbf774 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:24:27.395573 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: warnings.warn(msg) Nov 02 07:24:27.395573 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: Nov 02 07:24:27.396773 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a62cc23-e305-4982-84fe-2ee955cbf774 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:24:27.456858 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-6a62cc23-e305-4982-84fe-2ee955cbf774 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dd464b5ec0947dea32aff6e3dd9f6f7', 'request_id': 'req-6a62cc23-e305-4982-84fe-2ee955cbf774'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:24:27.469451 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-6a62cc23-e305-4982-84fe-2ee955cbf774 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dd464b5ec0947dea32aff6e3dd9f6f7', 'request_id': 'req-6a62cc23-e305-4982-84fe-2ee955cbf774'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:24:27.475383 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-6a62cc23-e305-4982-84fe-2ee955cbf774 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dd464b5ec0947dea32aff6e3dd9f6f7', 'request_id': 'req-6a62cc23-e305-4982-84fe-2ee955cbf774'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:24:27.505205 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-6a62cc23-e305-4982-84fe-2ee955cbf774 admin admin] The token cache is being invalidate because project 4dd464b5ec0947dea32aff6e3dd9f6f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:24:27.505484 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-6a62cc23-e305-4982-84fe-2ee955cbf774 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:24:27.505909 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-6a62cc23-e305-4982-84fe-2ee955cbf774 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:24:27.507352 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 598/1187] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:24:27 2022] DELETE /identity/v3/projects/4dd464b5ec0947dea32aff6e3dd9f6f7 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:24:36.897061 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=62 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:24:37.503081 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:25:55.913728 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c23988-893f-4ddc-9482-5c2e55eb45f9 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:55.914614 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c23988-893f-4ddc-9482-5c2e55eb45f9 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:55.914614 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c23988-893f-4ddc-9482-5c2e55eb45f9 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:55.916756 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 590/1188] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:25:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:25:55.924477 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55669315-8fd6-4e5f-92ac-60a16cdb3b90 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:55.924975 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55669315-8fd6-4e5f-92ac-60a16cdb3b90 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:55.925078 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55669315-8fd6-4e5f-92ac-60a16cdb3b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:56.023343 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-55669315-8fd6-4e5f-92ac-60a16cdb3b90 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:25:56.024565 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-55669315-8fd6-4e5f-92ac-60a16cdb3b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:25:56.154777 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 599/1189] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:25:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:25:56.310402 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c23988-893f-4ddc-9482-5c2e55eb45f9 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:25:56.314157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-09c23988-893f-4ddc-9482-5c2e55eb45f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:25:56.446626 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c23988-893f-4ddc-9482-5c2e55eb45f9 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:25:56.452219 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc490b-f039-4150-b4d2-762ef9389d63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:25:56.452819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc490b-f039-4150-b4d2-762ef9389d63 service nova] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:56.453089 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc490b-f039-4150-b4d2-762ef9389d63 service nova] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:56.453354 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc490b-f039-4150-b4d2-762ef9389d63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:56.461728 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc490b-f039-4150-b4d2-762ef9389d63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:25:56.463366 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc490b-f039-4150-b4d2-762ef9389d63 service nova] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:25:56.472794 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 591/1190] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:25:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:25:57.187409 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e51575-9156-4a85-9321-eb0d3f5faecc None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:57.187996 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e51575-9156-4a85-9321-eb0d3f5faecc None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:57.187996 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e51575-9156-4a85-9321-eb0d3f5faecc None None] PATH_INFO: `/v3` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:57.189613 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 600/1191] 10.222.0.40 () {54 vars in 925 bytes} [Wed Nov 2 07:25:57 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:25:57.196361 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84809320-f5f4-49eb-9caa-e7f85ea3fcd1 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:57.196817 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84809320-f5f4-49eb-9caa-e7f85ea3fcd1 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:57.196918 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84809320-f5f4-49eb-9caa-e7f85ea3fcd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:57.261777 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-84809320-f5f4-49eb-9caa-e7f85ea3fcd1 None None] MFA Rules not processed for user `932d3076fb4b426d8c83d02eb60d8063`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:25:57.263037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-84809320-f5f4-49eb-9caa-e7f85ea3fcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:25:57.334464 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:25:57.340005 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 592/1192] 10.222.0.40 () {58 vars in 1015 bytes} [Wed Nov 2 07:25:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:25:57.544608 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018733cc-0327-46ea-9691-d65d2439b04b None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:57.544844 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018733cc-0327-46ea-9691-d65d2439b04b None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:57.545020 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018733cc-0327-46ea-9691-d65d2439b04b None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:57.546407 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 601/1193] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:25:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:25:57.552429 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc8537-9cb0-4158-a863-c04bd32a02a2 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:57.552867 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc8537-9cb0-4158-a863-c04bd32a02a2 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:57.553311 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc8537-9cb0-4158-a863-c04bd32a02a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:57.582712 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-91dc8537-9cb0-4158-a863-c04bd32a02a2 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:25:57.583941 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-91dc8537-9cb0-4158-a863-c04bd32a02a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:25:57.592383 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 593/1194] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:25:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:25:57.746338 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018733cc-0327-46ea-9691-d65d2439b04b None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:25:57.764303 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018733cc-0327-46ea-9691-d65d2439b04b None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:25:57.770517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a23c1-5ed6-4b47-af62-fc628924a431 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:25:57.771152 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a23c1-5ed6-4b47-af62-fc628924a431 service nova] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:25:57.771428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a23c1-5ed6-4b47-af62-fc628924a431 service nova] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:25:57.771691 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a23c1-5ed6-4b47-af62-fc628924a431 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:25:57.780122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803a23c1-5ed6-4b47-af62-fc628924a431 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:25:57.781706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803a23c1-5ed6-4b47-af62-fc628924a431 service nova] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:25:57.791770 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 602/1195] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:25:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:26:06.124837 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df706cdd-3afa-4cd5-87b4-667db804fa14 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:26:06.125312 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df706cdd-3afa-4cd5-87b4-667db804fa14 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:26:06.125312 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df706cdd-3afa-4cd5-87b4-667db804fa14 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:26:06.126699 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 594/1196] 10.222.0.30 () {54 vars in 961 bytes} [Wed Nov 2 07:26:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:26:06.132230 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71bc89d-ae8f-4ea0-8993-83519cad4347 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:26:06.132706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71bc89d-ae8f-4ea0-8993-83519cad4347 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:26:06.132802 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71bc89d-ae8f-4ea0-8993-83519cad4347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:26:06.154293 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-e71bc89d-ae8f-4ea0-8993-83519cad4347 None None] MFA Rules not processed for user `932d3076fb4b426d8c83d02eb60d8063`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:26:06.155535 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-e71bc89d-ae8f-4ea0-8993-83519cad4347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:26:06.164982 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 603/1197] 10.222.0.30 () {58 vars in 1058 bytes} [Wed Nov 2 07:26:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:26:06.174990 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df706cdd-3afa-4cd5-87b4-667db804fa14 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:26:06.189968 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df706cdd-3afa-4cd5-87b4-667db804fa14 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:26:06.195655 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d01d0-e573-4eb4-89c3-ca121367e4a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932d3076fb4b426d8c83d02eb60d8063', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:26:06.196249 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d01d0-e573-4eb4-89c3-ca121367e4a6 service placement] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:26:06.196523 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d01d0-e573-4eb4-89c3-ca121367e4a6 service placement] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:26:06.196789 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d01d0-e573-4eb4-89c3-ca121367e4a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:26:06.205288 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d01d0-e573-4eb4-89c3-ca121367e4a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:26:06.207179 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d01d0-e573-4eb4-89c3-ca121367e4a6 service placement] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:26:06.216884 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 595/1198] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:26:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:26:09.255178 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61356c3-5cc7-4ea8-83fa-fff0552216c3 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:26:09.256117 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61356c3-5cc7-4ea8-83fa-fff0552216c3 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:26:09.256117 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61356c3-5cc7-4ea8-83fa-fff0552216c3 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:26:09.258181 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 604/1199] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:26:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:26:09.265860 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ae08e-9706-48d6-bd5d-55bdeb55a14f None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:26:09.266310 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ae08e-9706-48d6-bd5d-55bdeb55a14f None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:26:09.266409 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ae08e-9706-48d6-bd5d-55bdeb55a14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:26:09.286417 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-239ae08e-9706-48d6-bd5d-55bdeb55a14f None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:26:09.287640 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-239ae08e-9706-48d6-bd5d-55bdeb55a14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:26:09.295940 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 596/1200] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:26:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:26:09.463822 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61356c3-5cc7-4ea8-83fa-fff0552216c3 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:26:09.487467 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61356c3-5cc7-4ea8-83fa-fff0552216c3 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:26:09.494277 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f752c7-8328-42b9-92ef-f4486e4ec049 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:26:09.494921 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f752c7-8328-42b9-92ef-f4486e4ec049 service nova] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:26:09.495197 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f752c7-8328-42b9-92ef-f4486e4ec049 service nova] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:26:09.495464 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f752c7-8328-42b9-92ef-f4486e4ec049 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:26:09.504410 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f752c7-8328-42b9-92ef-f4486e4ec049 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:26:09.506067 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f752c7-8328-42b9-92ef-f4486e4ec049 service nova] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:26:09.516260 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 605/1201] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:26:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:26:10.564000 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42480d5f-6283-4515-b90d-1f508735a2f7 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:26:10.564952 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42480d5f-6283-4515-b90d-1f508735a2f7 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:26:10.564952 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42480d5f-6283-4515-b90d-1f508735a2f7 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:26:10.566712 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 597/1202] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 07:26:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:26:10.571935 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c4366-07f3-47e5-bc13-5604a88902bd None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:26:10.572424 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c4366-07f3-47e5-bc13-5604a88902bd None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:26:10.572424 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c4366-07f3-47e5-bc13-5604a88902bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:26:10.594628 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-1e2c4366-07f3-47e5-bc13-5604a88902bd None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:26:10.595824 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-1e2c4366-07f3-47e5-bc13-5604a88902bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:26:10.605234 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 606/1203] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 07:26:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:26:10.760247 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42480d5f-6283-4515-b90d-1f508735a2f7 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:26:10.779948 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42480d5f-6283-4515-b90d-1f508735a2f7 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:26:10.789797 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f98f5-005e-4104-8012-a8de07cdbfef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:26:10.790650 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f98f5-005e-4104-8012-a8de07cdbfef service nova] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:26:10.791006 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f98f5-005e-4104-8012-a8de07cdbfef service nova] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:26:10.791368 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f98f5-005e-4104-8012-a8de07cdbfef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:26:10.803364 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f98f5-005e-4104-8012-a8de07cdbfef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:26:10.804943 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f98f5-005e-4104-8012-a8de07cdbfef service nova] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:26:10.814727 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 598/1204] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:26:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:26:14.874495 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167d2ad-5e2e-4bc0-aab8-54bc5d387176 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:26:14.875094 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167d2ad-5e2e-4bc0-aab8-54bc5d387176 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:26:14.875094 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167d2ad-5e2e-4bc0-aab8-54bc5d387176 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:26:14.876321 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 607/1205] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:26:14.883373 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63710b31-f2d4-4d9b-91cd-bf557fe6f025 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:26:14.884122 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63710b31-f2d4-4d9b-91cd-bf557fe6f025 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:26:14.884225 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63710b31-f2d4-4d9b-91cd-bf557fe6f025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:26:14.916405 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-63710b31-f2d4-4d9b-91cd-bf557fe6f025 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:26:14.917620 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-63710b31-f2d4-4d9b-91cd-bf557fe6f025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:26:14.927066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 599/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:26:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:26:15.215010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167d2ad-5e2e-4bc0-aab8-54bc5d387176 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:26:15.218016 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-7167d2ad-5e2e-4bc0-aab8-54bc5d387176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:26:15.259545 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:26:15.306666 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167d2ad-5e2e-4bc0-aab8-54bc5d387176 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:26:15.310267 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6ee0e-bdcf-4ebb-84e9-13711ca5e8ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:26:15.310902 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6ee0e-bdcf-4ebb-84e9-13711ca5e8ac service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:26:15.311177 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6ee0e-bdcf-4ebb-84e9-13711ca5e8ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:26:15.311439 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6ee0e-bdcf-4ebb-84e9-13711ca5e8ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:26:15.320741 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a6ee0e-bdcf-4ebb-84e9-13711ca5e8ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:26:15.322260 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a6ee0e-bdcf-4ebb-84e9-13711ca5e8ac service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:26:15.332385 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 608/1207] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:26:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:26:17.484165 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6528c-0436-4722-b51b-21a7a9a6bff1 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:26:17.484777 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6528c-0436-4722-b51b-21a7a9a6bff1 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:26:17.484777 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6528c-0436-4722-b51b-21a7a9a6bff1 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:26:17.486099 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 600/1208] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:26:17.493469 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ceee6-3bf8-408b-b131-d6bfc6a393c7 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:26:17.493896 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ceee6-3bf8-408b-b131-d6bfc6a393c7 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:26:17.494010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ceee6-3bf8-408b-b131-d6bfc6a393c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:26:17.516007 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-375ceee6-3bf8-408b-b131-d6bfc6a393c7 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:26:17.517228 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-375ceee6-3bf8-408b-b131-d6bfc6a393c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:26:17.527204 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 609/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:26:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:26:17.811966 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc6528c-0436-4722-b51b-21a7a9a6bff1 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:26:17.826485 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc6528c-0436-4722-b51b-21a7a9a6bff1 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:26:17.830514 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fcca60-92aa-454b-b9f5-bfdad49554e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:26:17.831164 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcca60-92aa-454b-b9f5-bfdad49554e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:26:17.831439 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcca60-92aa-454b-b9f5-bfdad49554e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:26:17.831712 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcca60-92aa-454b-b9f5-bfdad49554e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:26:17.841024 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fcca60-92aa-454b-b9f5-bfdad49554e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:26:17.842447 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fcca60-92aa-454b-b9f5-bfdad49554e9 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:26:17.852131 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 601/1210] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:26:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:26:21.749767 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0803e7-dce3-4a1c-87b9-bc6169f47f3b None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:26:21.750336 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0803e7-dce3-4a1c-87b9-bc6169f47f3b None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:26:21.750336 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0803e7-dce3-4a1c-87b9-bc6169f47f3b None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:26:21.752358 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 610/1211] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:26:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:26:21.759368 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997f2fa-164c-463a-a263-f957229cbb14 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:26:21.759836 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997f2fa-164c-463a-a263-f957229cbb14 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:26:21.759929 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997f2fa-164c-463a-a263-f957229cbb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:26:21.780755 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-c997f2fa-164c-463a-a263-f957229cbb14 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:26:21.781943 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-c997f2fa-164c-463a-a263-f957229cbb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:26:21.791291 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 602/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:26:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:26:22.092598 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0803e7-dce3-4a1c-87b9-bc6169f47f3b None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:26:22.094527 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-4f0803e7-dce3-4a1c-87b9-bc6169f47f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:26:22.177770 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0803e7-dce3-4a1c-87b9-bc6169f47f3b None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:26:22.181875 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7772403a-c635-48cd-b668-cd7fd1ce9686 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:26:22.182482 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7772403a-c635-48cd-b668-cd7fd1ce9686 service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:26:22.182789 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7772403a-c635-48cd-b668-cd7fd1ce9686 service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:26:22.183061 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7772403a-c635-48cd-b668-cd7fd1ce9686 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:26:22.192361 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7772403a-c635-48cd-b668-cd7fd1ce9686 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:26:22.193934 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7772403a-c635-48cd-b668-cd7fd1ce9686 service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:26:22.203937 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 611/1213] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:26:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:26:24.345115 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d5150-d023-4015-89f9-9a1652309dfb None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:26:24.346039 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d5150-d023-4015-89f9-9a1652309dfb None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:26:24.346039 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d5150-d023-4015-89f9-9a1652309dfb None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:26:24.348193 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 603/1214] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 07:26:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:26:24.353774 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681f997-de7a-4142-8b78-673d61cb14b2 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:26:24.354226 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681f997-de7a-4142-8b78-673d61cb14b2 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:26:24.354321 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681f997-de7a-4142-8b78-673d61cb14b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:26:24.375288 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-d681f997-de7a-4142-8b78-673d61cb14b2 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:26:24.376490 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-d681f997-de7a-4142-8b78-673d61cb14b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:26:24.384990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 612/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 07:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:26:24.684410 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d5150-d023-4015-89f9-9a1652309dfb None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:26:24.686690 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-9f5d5150-d023-4015-89f9-9a1652309dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:26:24.702014 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d5150-d023-4015-89f9-9a1652309dfb None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:26:24.705849 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b2730-ad45-4575-91ab-5432daf21f88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:26:24.706439 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b2730-ad45-4575-91ab-5432daf21f88 service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:26:24.706711 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b2730-ad45-4575-91ab-5432daf21f88 service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:26:24.706997 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b2730-ad45-4575-91ab-5432daf21f88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:26:24.716245 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b2730-ad45-4575-91ab-5432daf21f88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:26:24.717657 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b2730-ad45-4575-91ab-5432daf21f88 service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:26:24.727420 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 604/1216] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:26:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:26:34.370169 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=63 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:26:34.723169 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=37 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:50.228709 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5a660-a7d1-48d7-9d8a-26bbe060625a None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:50.230888 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5a660-a7d1-48d7-9d8a-26bbe060625a None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:50.230888 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5a660-a7d1-48d7-9d8a-26bbe060625a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:50.260661 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-f8d5a660-a7d1-48d7-9d8a-26bbe060625a None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:50.261904 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-f8d5a660-a7d1-48d7-9d8a-26bbe060625a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:50.275481 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 613/1217] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:50.284633 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e50c3-5518-4ddd-a8be-fc1c028aca23 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:50.285718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e50c3-5518-4ddd-a8be-fc1c028aca23 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:50.286199 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e50c3-5518-4ddd-a8be-fc1c028aca23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:50.316580 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-344e50c3-5518-4ddd-a8be-fc1c028aca23 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:50.318330 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-344e50c3-5518-4ddd-a8be-fc1c028aca23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:50.330929 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 605/1218] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:50.337448 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5a660-a7d1-48d7-9d8a-26bbe060625a None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:50.351046 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5a660-a7d1-48d7-9d8a-26bbe060625a None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:50.355869 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc0ef0-6a68-4102-95d4-f2af269b9f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:50.356614 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc0ef0-6a68-4102-95d4-f2af269b9f94 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:50.356891 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc0ef0-6a68-4102-95d4-f2af269b9f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:50.357162 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc0ef0-6a68-4102-95d4-f2af269b9f94 admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:50.357806 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc0ef0-6a68-4102-95d4-f2af269b9f94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:27:50.358103 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc0ef0-6a68-4102-95d4-f2af269b9f94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:50.359734 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc0ef0-6a68-4102-95d4-f2af269b9f94 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:50.380574 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 614/1219] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:27:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:27:50.389427 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e50c3-5518-4ddd-a8be-fc1c028aca23 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:50.394262 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b34e7-815e-4f20-a2ad-0ae239cab5e2 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:50.394577 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b34e7-815e-4f20-a2ad-0ae239cab5e2 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:50.394852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b34e7-815e-4f20-a2ad-0ae239cab5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:50.404221 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e50c3-5518-4ddd-a8be-fc1c028aca23 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:50.412856 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c8d044-8d0f-4cee-bd45-18e67edcc868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:50.413850 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8d044-8d0f-4cee-bd45-18e67edcc868 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:50.414299 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8d044-8d0f-4cee-bd45-18e67edcc868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:50.414792 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8d044-8d0f-4cee-bd45-18e67edcc868 admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:50.415893 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c8d044-8d0f-4cee-bd45-18e67edcc868 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:50.418213 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c8d044-8d0f-4cee-bd45-18e67edcc868 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:50.426679 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-125b34e7-815e-4f20-a2ad-0ae239cab5e2 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:50.426679 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-125b34e7-815e-4f20-a2ad-0ae239cab5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:50.446679 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 615/1220] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:27:50.452291 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff789bf3-b717-4a37-a97a-102ce00ae28b None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:50.452577 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff789bf3-b717-4a37-a97a-102ce00ae28b None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:50.452854 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff789bf3-b717-4a37-a97a-102ce00ae28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:50.491653 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-ff789bf3-b717-4a37-a97a-102ce00ae28b None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:50.492794 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-ff789bf3-b717-4a37-a97a-102ce00ae28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:50.492970 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 606/1221] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:27:50 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:50.495007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78375c2-408e-4c85-bc66-91627cec5d48 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:50.495459 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78375c2-408e-4c85-bc66-91627cec5d48 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:50.495867 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78375c2-408e-4c85-bc66-91627cec5d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:50.531889 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-a78375c2-408e-4c85-bc66-91627cec5d48 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:50.533506 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-a78375c2-408e-4c85-bc66-91627cec5d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:50.634288 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 616/1222] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:27:50.635041 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff789bf3-b717-4a37-a97a-102ce00ae28b None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:50.644322 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff789bf3-b717-4a37-a97a-102ce00ae28b None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:50.648888 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177b6cf-498f-4d51-92a0-d98296568e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:50.649532 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177b6cf-498f-4d51-92a0-d98296568e03 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:50.649799 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177b6cf-498f-4d51-92a0-d98296568e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:50.650189 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177b6cf-498f-4d51-92a0-d98296568e03 admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:50.650860 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177b6cf-498f-4d51-92a0-d98296568e03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:50.657346 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 607/1223] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:50.657527 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177b6cf-498f-4d51-92a0-d98296568e03 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:50.657623 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742011b2-919c-4ebc-b3d2-7344e1ebe593 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:50.657623 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742011b2-919c-4ebc-b3d2-7344e1ebe593 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:50.657623 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742011b2-919c-4ebc-b3d2-7344e1ebe593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:50.674521 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-742011b2-919c-4ebc-b3d2-7344e1ebe593 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:50.677905 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 617/1224] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:27:50 2022] POST /identity/v3/users => generated 293 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:50.678063 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-742011b2-919c-4ebc-b3d2-7344e1ebe593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:50.678151 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8cd79-6b9e-4415-85ec-17fdadf5f457 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:50.678151 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8cd79-6b9e-4415-85ec-17fdadf5f457 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:50.678151 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8cd79-6b9e-4415-85ec-17fdadf5f457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:50.684052 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 608/1225] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:50.685349 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2ad15-3bf4-413e-81a7-4cd3584a4cda None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:50.685627 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2ad15-3bf4-413e-81a7-4cd3584a4cda None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:50.685900 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2ad15-3bf4-413e-81a7-4cd3584a4cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:50.697116 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-83b8cd79-6b9e-4415-85ec-17fdadf5f457 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:50.698329 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-83b8cd79-6b9e-4415-85ec-17fdadf5f457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:50.706275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-adc2ad15-3bf4-413e-81a7-4cd3584a4cda None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:50.707463 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-adc2ad15-3bf4-413e-81a7-4cd3584a4cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:50.708650 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 618/1226] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:50.709862 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea4250-c60a-4864-9636-c1f2233f0b06 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:50.710131 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea4250-c60a-4864-9636-c1f2233f0b06 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:50.710397 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea4250-c60a-4864-9636-c1f2233f0b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:50.718765 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 609/1227] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:50.718765 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8cff5-57fa-4ec8-8a36-c6e1de4749a5 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:50.718765 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8cff5-57fa-4ec8-8a36-c6e1de4749a5 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:50.718765 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8cff5-57fa-4ec8-8a36-c6e1de4749a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:50.734070 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-30ea4250-c60a-4864-9636-c1f2233f0b06 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:50.735297 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-30ea4250-c60a-4864-9636-c1f2233f0b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:50.741110 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-e4c8cff5-57fa-4ec8-8a36-c6e1de4749a5 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:50.742548 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-e4c8cff5-57fa-4ec8-8a36-c6e1de4749a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:50.744049 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 619/1228] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:50.745035 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea4250-c60a-4864-9636-c1f2233f0b06 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:50.755489 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea4250-c60a-4864-9636-c1f2233f0b06 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:50.755683 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 610/1229] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:50.756900 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04e4d7-644b-4248-8522-8c0fe9fbe83a None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:50.757178 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04e4d7-644b-4248-8522-8c0fe9fbe83a None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:50.757439 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04e4d7-644b-4248-8522-8c0fe9fbe83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:50.765695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243c756-39eb-4dc4-92a8-9b8967aa2501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:50.766337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243c756-39eb-4dc4-92a8-9b8967aa2501 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:50.766615 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243c756-39eb-4dc4-92a8-9b8967aa2501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:50.766923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243c756-39eb-4dc4-92a8-9b8967aa2501 admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:50.767567 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a243c756-39eb-4dc4-92a8-9b8967aa2501 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:27:50.767953 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a243c756-39eb-4dc4-92a8-9b8967aa2501 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:50.769459 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a243c756-39eb-4dc4-92a8-9b8967aa2501 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:50.779914 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-4b04e4d7-644b-4248-8522-8c0fe9fbe83a None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:50.781124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-4b04e4d7-644b-4248-8522-8c0fe9fbe83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:50.781593 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 620/1230] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:27:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:27:50.786892 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243c756-39eb-4dc4-92a8-9b8967aa2501 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:50.792108 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243c756-39eb-4dc4-92a8-9b8967aa2501 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:50.794688 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 611/1231] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:50.795915 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7edbf4-a4bb-49dc-9fbd-675cdf94acc0 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:50.796202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7edbf4-a4bb-49dc-9fbd-675cdf94acc0 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:50.796468 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7edbf4-a4bb-49dc-9fbd-675cdf94acc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:50.796644 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b96cc0-a46d-4672-affc-26dc6a07b161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:50.797223 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b96cc0-a46d-4672-affc-26dc6a07b161 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:50.797493 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b96cc0-a46d-4672-affc-26dc6a07b161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:50.797758 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b96cc0-a46d-4672-affc-26dc6a07b161 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:50.798384 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b96cc0-a46d-4672-affc-26dc6a07b161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:50.800060 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b96cc0-a46d-4672-affc-26dc6a07b161 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:50.812897 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 621/1232] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:50.814107 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888b02d-d58c-4a32-96f0-0587aae75464 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:50.814387 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888b02d-d58c-4a32-96f0-0587aae75464 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:50.814647 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888b02d-d58c-4a32-96f0-0587aae75464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:50.817193 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-5e7edbf4-a4bb-49dc-9fbd-675cdf94acc0 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:50.818454 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-5e7edbf4-a4bb-49dc-9fbd-675cdf94acc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:50.827159 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 612/1233] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:50.828376 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f15b5-c123-4718-b1ca-034f3b13c838 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:50.828658 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f15b5-c123-4718-b1ca-034f3b13c838 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:50.828928 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f15b5-c123-4718-b1ca-034f3b13c838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:50.837421 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-6888b02d-d58c-4a32-96f0-0587aae75464 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:50.838815 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-6888b02d-d58c-4a32-96f0-0587aae75464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:50.847375 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 622/1234] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:50.847998 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-a06f15b5-c123-4718-b1ca-034f3b13c838 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:50.848587 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71312c-00ac-428d-bf1a-ef5c15ad6c53 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:50.848881 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71312c-00ac-428d-bf1a-ef5c15ad6c53 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:50.849168 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-a06f15b5-c123-4718-b1ca-034f3b13c838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:50.849284 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71312c-00ac-428d-bf1a-ef5c15ad6c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:50.857602 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 613/1235] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:50.858873 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ddaddf-ab84-4853-9cba-3f6ceaeac4c8 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:50.859109 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ddaddf-ab84-4853-9cba-3f6ceaeac4c8 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:50.859374 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ddaddf-ab84-4853-9cba-3f6ceaeac4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:50.870607 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-dd71312c-00ac-428d-bf1a-ef5c15ad6c53 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:50.871858 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-dd71312c-00ac-428d-bf1a-ef5c15ad6c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:50.883482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-77ddaddf-ab84-4853-9cba-3f6ceaeac4c8 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:50.883482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-77ddaddf-ab84-4853-9cba-3f6ceaeac4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:50.883762 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 623/1236] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:50.883762 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71312c-00ac-428d-bf1a-ef5c15ad6c53 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:50.890759 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 614/1237] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:50.891585 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ddaddf-ab84-4853-9cba-3f6ceaeac4c8 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:50.892726 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71312c-00ac-428d-bf1a-ef5c15ad6c53 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:50.897552 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6eed4-502f-4fc2-ad54-84a2083f6c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:50.898161 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6eed4-502f-4fc2-ad54-84a2083f6c43 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:50.898983 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6eed4-502f-4fc2-ad54-84a2083f6c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:50.899261 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6eed4-502f-4fc2-ad54-84a2083f6c43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:50.899963 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb6eed4-502f-4fc2-ad54-84a2083f6c43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:50.901006 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: WARNING py.warnings [None req-0eb6eed4-502f-4fc2-ad54-84a2083f6c43 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:27:50.901006 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: warnings.warn(msg) Nov 02 07:27:50.901006 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: Nov 02 07:27:50.901297 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ddaddf-ab84-4853-9cba-3f6ceaeac4c8 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:50.902200 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb6eed4-502f-4fc2-ad54-84a2083f6c43 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:50.905870 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193b7fe-2e39-45e4-bbb9-98683acc40e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:50.906515 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193b7fe-2e39-45e4-bbb9-98683acc40e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:50.907026 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193b7fe-2e39-45e4-bbb9-98683acc40e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:50.907303 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193b7fe-2e39-45e4-bbb9-98683acc40e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:50.908301 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a193b7fe-2e39-45e4-bbb9-98683acc40e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:27:50.908599 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a193b7fe-2e39-45e4-bbb9-98683acc40e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:50.909613 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: WARNING py.warnings [None req-a193b7fe-2e39-45e4-bbb9-98683acc40e3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:27:50.909613 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: warnings.warn(msg) Nov 02 07:27:50.909613 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: Nov 02 07:27:50.910480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a193b7fe-2e39-45e4-bbb9-98683acc40e3 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:50.923455 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 615/1238] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:27:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:27:50.924773 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193b7fe-2e39-45e4-bbb9-98683acc40e3 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:50.932970 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 624/1239] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:27:50 2022] POST /identity/v3/projects => generated 403 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:50.934127 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6eed4-502f-4fc2-ad54-84a2083f6c43 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:50.937429 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193b7fe-2e39-45e4-bbb9-98683acc40e3 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:50.984854 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:50.999327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04faf09b-8e3e-4e76-8bbd-d0264c43157b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:50.999876 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04faf09b-8e3e-4e76-8bbd-d0264c43157b admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.000149 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04faf09b-8e3e-4e76-8bbd-d0264c43157b admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.000416 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04faf09b-8e3e-4e76-8bbd-d0264c43157b admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.001052 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04faf09b-8e3e-4e76-8bbd-d0264c43157b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.002538 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6eed4-502f-4fc2-ad54-84a2083f6c43 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.002832 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04faf09b-8e3e-4e76-8bbd-d0264c43157b admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.007583 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a1638-11bb-4a11-a413-2262de4c1d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.008193 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a1638-11bb-4a11-a413-2262de4c1d29 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.008472 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a1638-11bb-4a11-a413-2262de4c1d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.008772 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a1638-11bb-4a11-a413-2262de4c1d29 admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.009416 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10a1638-11bb-4a11-a413-2262de4c1d29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:27:51.009718 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10a1638-11bb-4a11-a413-2262de4c1d29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.011370 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10a1638-11bb-4a11-a413-2262de4c1d29 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.015500 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:51.020634 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 616/1240] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:51.021525 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04faf09b-8e3e-4e76-8bbd-d0264c43157b admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.023267 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 625/1241] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:27:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:27:51.024041 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a1638-11bb-4a11-a413-2262de4c1d29 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.033796 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a1638-11bb-4a11-a413-2262de4c1d29 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.038446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e2417-eab6-42fb-a12d-0fcb2b6387c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.039068 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e2417-eab6-42fb-a12d-0fcb2b6387c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.039347 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e2417-eab6-42fb-a12d-0fcb2b6387c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.039616 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e2417-eab6-42fb-a12d-0fcb2b6387c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.040280 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322e2417-eab6-42fb-a12d-0fcb2b6387c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:27:51.040415 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04faf09b-8e3e-4e76-8bbd-d0264c43157b admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.040585 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322e2417-eab6-42fb-a12d-0fcb2b6387c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.042200 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322e2417-eab6-42fb-a12d-0fcb2b6387c4 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.045080 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5d694-3821-4180-b086-e3f16f99de20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.045746 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5d694-3821-4180-b086-e3f16f99de20 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.046085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5d694-3821-4180-b086-e3f16f99de20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.046431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5d694-3821-4180-b086-e3f16f99de20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.047216 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f5d694-3821-4180-b086-e3f16f99de20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:27:51.047517 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f5d694-3821-4180-b086-e3f16f99de20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.049091 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f5d694-3821-4180-b086-e3f16f99de20 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.054384 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 626/1242] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:27:51.055222 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e2417-eab6-42fb-a12d-0fcb2b6387c4 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.066550 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 617/1243] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:27:51.066550 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5d694-3821-4180-b086-e3f16f99de20 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.066863 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e2417-eab6-42fb-a12d-0fcb2b6387c4 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.070975 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c0229a-2dd2-4b85-b63f-866efc659e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.071671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0229a-2dd2-4b85-b63f-866efc659e45 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.071947 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0229a-2dd2-4b85-b63f-866efc659e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.072217 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0229a-2dd2-4b85-b63f-866efc659e45 admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.072857 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c0229a-2dd2-4b85-b63f-866efc659e45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:27:51.073187 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c0229a-2dd2-4b85-b63f-866efc659e45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.074823 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c0229a-2dd2-4b85-b63f-866efc659e45 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.075965 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5d694-3821-4180-b086-e3f16f99de20 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.080637 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c67b3-3eb2-4068-9f1e-00aeb5f7ef63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.081227 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c67b3-3eb2-4068-9f1e-00aeb5f7ef63 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.081475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c67b3-3eb2-4068-9f1e-00aeb5f7ef63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.081737 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c67b3-3eb2-4068-9f1e-00aeb5f7ef63 admin admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.082364 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939c67b3-3eb2-4068-9f1e-00aeb5f7ef63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:27:51.082682 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939c67b3-3eb2-4068-9f1e-00aeb5f7ef63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.084264 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939c67b3-3eb2-4068-9f1e-00aeb5f7ef63 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.087521 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 627/1244] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:27:51.095327 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c0229a-2dd2-4b85-b63f-866efc659e45 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.096269 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 618/1245] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:27:51.096862 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c67b3-3eb2-4068-9f1e-00aeb5f7ef63 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.105841 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c0229a-2dd2-4b85-b63f-866efc659e45 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.108252 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c67b3-3eb2-4068-9f1e-00aeb5f7ef63 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.111066 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99e80a-d116-4586-965e-60b305819a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.111663 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99e80a-d116-4586-965e-60b305819a44 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.111934 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99e80a-d116-4586-965e-60b305819a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.112199 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99e80a-d116-4586-965e-60b305819a44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.112724 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d621cf-b043-4ac2-8306-64ae5f46c1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.112847 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c99e80a-d116-4586-965e-60b305819a44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.113311 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d621cf-b043-4ac2-8306-64ae5f46c1b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.113583 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d621cf-b043-4ac2-8306-64ae5f46c1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.113851 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d621cf-b043-4ac2-8306-64ae5f46c1b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.114465 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d621cf-b043-4ac2-8306-64ae5f46c1b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.114768 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c99e80a-d116-4586-965e-60b305819a44 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.115551 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: WARNING py.warnings [None req-e8d621cf-b043-4ac2-8306-64ae5f46c1b4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:27:51.115551 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: warnings.warn(msg) Nov 02 07:27:51.115551 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: Nov 02 07:27:51.116740 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d621cf-b043-4ac2-8306-64ae5f46c1b4 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.141405 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 619/1246] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:27:51 2022] POST /identity/v3/users => generated 302 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:51.143679 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d621cf-b043-4ac2-8306-64ae5f46c1b4 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.144476 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 628/1247] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:27:51 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:51.145294 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99e80a-d116-4586-965e-60b305819a44 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.153795 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d621cf-b043-4ac2-8306-64ae5f46c1b4 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.208882 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99e80a-d116-4586-965e-60b305819a44 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.210283 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a2ba6c-3451-4ae8-a1bb-19fa102cfc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.210983 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2ba6c-3451-4ae8-a1bb-19fa102cfc7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.211263 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2ba6c-3451-4ae8-a1bb-19fa102cfc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.211529 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2ba6c-3451-4ae8-a1bb-19fa102cfc7f admin admin] PATH_INFO: `/v3/projects/04d1043eb7464a8987352ab1714424fc/users/53ce525fe4e04ea1a27fce3227ef7e40/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.214017 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80271a9a-f3a8-4250-8e63-846968c43054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.214645 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80271a9a-f3a8-4250-8e63-846968c43054 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.214935 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80271a9a-f3a8-4250-8e63-846968c43054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.215207 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80271a9a-f3a8-4250-8e63-846968c43054 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.215835 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80271a9a-f3a8-4250-8e63-846968c43054 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.217768 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80271a9a-f3a8-4250-8e63-846968c43054 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.226898 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a2ba6c-3451-4ae8-a1bb-19fa102cfc7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=04d1043eb7464a8987352ab1714424fc, user_id=53ce525fe4e04ea1a27fce3227ef7e40, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.230684 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a2ba6c-3451-4ae8-a1bb-19fa102cfc7f admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.246116 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 620/1248] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:27:51 2022] PUT /identity/v3/projects/04d1043eb7464a8987352ab1714424fc/users/53ce525fe4e04ea1a27fce3227ef7e40/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:51.249570 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 629/1249] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:27:51 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:51.249570 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80271a9a-f3a8-4250-8e63-846968c43054 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.256470 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a2ba6c-3451-4ae8-a1bb-19fa102cfc7f admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.315637 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80271a9a-f3a8-4250-8e63-846968c43054 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.320698 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b625c68-7dde-4126-ba2e-1653b39ecd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.321384 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b625c68-7dde-4126-ba2e-1653b39ecd3d admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.321704 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b625c68-7dde-4126-ba2e-1653b39ecd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.321986 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b625c68-7dde-4126-ba2e-1653b39ecd3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.322564 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a2ba6c-3451-4ae8-a1bb-19fa102cfc7f admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.323107 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b625c68-7dde-4126-ba2e-1653b39ecd3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.324510 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b625c68-7dde-4126-ba2e-1653b39ecd3d admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.327333 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197f859-5d6a-40e5-a6df-86dbf3da1b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.327941 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197f859-5d6a-40e5-a6df-86dbf3da1b7e admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.328208 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197f859-5d6a-40e5-a6df-86dbf3da1b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.328496 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197f859-5d6a-40e5-a6df-86dbf3da1b7e admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.329114 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197f859-5d6a-40e5-a6df-86dbf3da1b7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.331035 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197f859-5d6a-40e5-a6df-86dbf3da1b7e admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.347374 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:51.359097 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 630/1250] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:27:51 2022] POST /identity/v3/projects => generated 405 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:51.360027 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 621/1251] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:27:51 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:51.360738 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197f859-5d6a-40e5-a6df-86dbf3da1b7e admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.362928 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b625c68-7dde-4126-ba2e-1653b39ecd3d admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.390007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:51.427794 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b625c68-7dde-4126-ba2e-1653b39ecd3d admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.428235 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197f859-5d6a-40e5-a6df-86dbf3da1b7e admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.432528 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20b8ef-55ab-4972-9736-38fe300b4cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.433139 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20b8ef-55ab-4972-9736-38fe300b4cbe admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.433272 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5629762-8eed-4966-863a-cfe60d7864c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.433422 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20b8ef-55ab-4972-9736-38fe300b4cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.433688 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20b8ef-55ab-4972-9736-38fe300b4cbe admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.433793 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5629762-8eed-4966-863a-cfe60d7864c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.434011 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5629762-8eed-4966-863a-cfe60d7864c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.434279 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5629762-8eed-4966-863a-cfe60d7864c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.434389 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc20b8ef-55ab-4972-9736-38fe300b4cbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.434941 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5629762-8eed-4966-863a-cfe60d7864c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.436240 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc20b8ef-55ab-4972-9736-38fe300b4cbe admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.436867 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5629762-8eed-4966-863a-cfe60d7864c1 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.464528 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 631/1252] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:27:51 2022] POST /identity/v3/projects => generated 365 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:51.465603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20b8ef-55ab-4972-9736-38fe300b4cbe admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.468295 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 622/1253] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:27:51 2022] POST /identity/v3/projects => generated 393 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:51.469192 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5629762-8eed-4966-863a-cfe60d7864c1 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.528388 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20b8ef-55ab-4972-9736-38fe300b4cbe admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.530970 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5629762-8eed-4966-863a-cfe60d7864c1 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.533592 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c94299-e833-4c36-8f78-3a77b64821ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.534202 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c94299-e833-4c36-8f78-3a77b64821ad admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.534474 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c94299-e833-4c36-8f78-3a77b64821ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.534768 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c94299-e833-4c36-8f78-3a77b64821ad admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.535393 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c94299-e833-4c36-8f78-3a77b64821ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.535514 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f7eee4-d7df-44f2-9ae6-4aa47e4a2b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.536077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f7eee4-d7df-44f2-9ae6-4aa47e4a2b6d admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.536379 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f7eee4-d7df-44f2-9ae6-4aa47e4a2b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.536644 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f7eee4-d7df-44f2-9ae6-4aa47e4a2b6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.537279 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f7eee4-d7df-44f2-9ae6-4aa47e4a2b6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.537388 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c94299-e833-4c36-8f78-3a77b64821ad admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.538909 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f7eee4-d7df-44f2-9ae6-4aa47e4a2b6d admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.549288 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 623/1254] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:51.550395 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f7eee4-d7df-44f2-9ae6-4aa47e4a2b6d admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.559992 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 632/1255] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:27:51 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:51.561248 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771d6f0-b547-47b2-8c78-3aee20719464 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.561622 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f7eee4-d7df-44f2-9ae6-4aa47e4a2b6d admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.561747 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771d6f0-b547-47b2-8c78-3aee20719464 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.562039 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771d6f0-b547-47b2-8c78-3aee20719464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.566401 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22842500-9492-4532-9adc-41025b382244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.567017 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22842500-9492-4532-9adc-41025b382244 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.567291 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22842500-9492-4532-9adc-41025b382244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.567556 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22842500-9492-4532-9adc-41025b382244 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.568173 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22842500-9492-4532-9adc-41025b382244 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.570125 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22842500-9492-4532-9adc-41025b382244 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.590900 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-6771d6f0-b547-47b2-8c78-3aee20719464 None None] MFA Rules not processed for user `53ce525fe4e04ea1a27fce3227ef7e40`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:51.592146 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-6771d6f0-b547-47b2-8c78-3aee20719464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:51.594412 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 624/1256] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:27:51 2022] POST /identity/v3/users => generated 288 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:51.595381 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22842500-9492-4532-9adc-41025b382244 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.606393 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22842500-9492-4532-9adc-41025b382244 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.611437 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62279ac-0a08-4c8b-bdf1-395307f9a7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.612049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62279ac-0a08-4c8b-bdf1-395307f9a7ac admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.612328 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62279ac-0a08-4c8b-bdf1-395307f9a7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.612594 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62279ac-0a08-4c8b-bdf1-395307f9a7ac admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.613217 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62279ac-0a08-4c8b-bdf1-395307f9a7ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.615185 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62279ac-0a08-4c8b-bdf1-395307f9a7ac admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.640622 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 625/1257] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:27:51 2022] POST /identity/v3/users => generated 303 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:51.641740 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62279ac-0a08-4c8b-bdf1-395307f9a7ac admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.651856 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62279ac-0a08-4c8b-bdf1-395307f9a7ac admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.656666 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386db1a-13a0-4fc5-a7ea-11f73f7fcebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.657251 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386db1a-13a0-4fc5-a7ea-11f73f7fcebf admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.657518 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386db1a-13a0-4fc5-a7ea-11f73f7fcebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.657783 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386db1a-13a0-4fc5-a7ea-11f73f7fcebf admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.658393 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4386db1a-13a0-4fc5-a7ea-11f73f7fcebf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.660317 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4386db1a-13a0-4fc5-a7ea-11f73f7fcebf admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.663975 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 633/1258] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:27:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:51.664824 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771d6f0-b547-47b2-8c78-3aee20719464 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.671774 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:51.676080 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771d6f0-b547-47b2-8c78-3aee20719464 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.681785 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89367d3-a816-42ea-b4f1-54779e1a9d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.682503 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89367d3-a816-42ea-b4f1-54779e1a9d4e admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.682999 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89367d3-a816-42ea-b4f1-54779e1a9d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.683440 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89367d3-a816-42ea-b4f1-54779e1a9d4e admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.684297 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89367d3-a816-42ea-b4f1-54779e1a9d4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.685191 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 626/1259] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:27:51 2022] POST /identity/v3/users => generated 292 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:51.686017 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386db1a-13a0-4fc5-a7ea-11f73f7fcebf admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.689651 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89367d3-a816-42ea-b4f1-54779e1a9d4e admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.695242 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386db1a-13a0-4fc5-a7ea-11f73f7fcebf admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.700180 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e5a64-2e48-45f1-a4bd-f01620133543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.700784 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e5a64-2e48-45f1-a4bd-f01620133543 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.701061 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e5a64-2e48-45f1-a4bd-f01620133543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.701359 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e5a64-2e48-45f1-a4bd-f01620133543 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.702089 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86e5a64-2e48-45f1-a4bd-f01620133543 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.703962 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86e5a64-2e48-45f1-a4bd-f01620133543 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.713174 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 634/1260] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:27:51 2022] POST /identity/v3/users => generated 283 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:51.715820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89367d3-a816-42ea-b4f1-54779e1a9d4e admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.726744 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89367d3-a816-42ea-b4f1-54779e1a9d4e admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.731094 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 627/1261] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:27:51 2022] POST /identity/v3/users => generated 297 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:51.731094 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e5a64-2e48-45f1-a4bd-f01620133543 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.731773 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60755061-8fcd-4f50-84e7-2f5d0df77c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.732388 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60755061-8fcd-4f50-84e7-2f5d0df77c3d admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.732664 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60755061-8fcd-4f50-84e7-2f5d0df77c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.732935 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60755061-8fcd-4f50-84e7-2f5d0df77c3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.733605 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60755061-8fcd-4f50-84e7-2f5d0df77c3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.735331 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60755061-8fcd-4f50-84e7-2f5d0df77c3d admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.739137 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e5a64-2e48-45f1-a4bd-f01620133543 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.744607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44e7aa-0c36-48b2-b3b9-aa2c7e927dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.745213 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44e7aa-0c36-48b2-b3b9-aa2c7e927dbd admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.745597 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44e7aa-0c36-48b2-b3b9-aa2c7e927dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.745874 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44e7aa-0c36-48b2-b3b9-aa2c7e927dbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.746509 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44e7aa-0c36-48b2-b3b9-aa2c7e927dbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.746631 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 635/1262] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:51.747491 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60755061-8fcd-4f50-84e7-2f5d0df77c3d admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.751429 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44e7aa-0c36-48b2-b3b9-aa2c7e927dbd admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.757285 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60755061-8fcd-4f50-84e7-2f5d0df77c3d admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.762311 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979eb7c4-d90c-4408-8474-0b89da1b5f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.762602 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 628/1263] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:51.762920 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979eb7c4-d90c-4408-8474-0b89da1b5f5b admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.763196 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979eb7c4-d90c-4408-8474-0b89da1b5f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.763422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44e7aa-0c36-48b2-b3b9-aa2c7e927dbd admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.763524 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979eb7c4-d90c-4408-8474-0b89da1b5f5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.764079 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979eb7c4-d90c-4408-8474-0b89da1b5f5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.765644 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979eb7c4-d90c-4408-8474-0b89da1b5f5b admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.772618 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44e7aa-0c36-48b2-b3b9-aa2c7e927dbd admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.775558 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 636/1264] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:51.776363 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979eb7c4-d90c-4408-8474-0b89da1b5f5b admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.777480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de16fbf-dec6-499b-aeeb-913454ad51c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.778066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de16fbf-dec6-499b-aeeb-913454ad51c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.778337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de16fbf-dec6-499b-aeeb-913454ad51c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.778801 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de16fbf-dec6-499b-aeeb-913454ad51c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.779261 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de16fbf-dec6-499b-aeeb-913454ad51c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.780821 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de16fbf-dec6-499b-aeeb-913454ad51c4 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.787118 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979eb7c4-d90c-4408-8474-0b89da1b5f5b admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.793792 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 629/1265] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:51.793938 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfafd99-4ea7-4e4c-a3d8-970c0c6f54a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.794049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de16fbf-dec6-499b-aeeb-913454ad51c4 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.794148 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfafd99-4ea7-4e4c-a3d8-970c0c6f54a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.794148 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfafd99-4ea7-4e4c-a3d8-970c0c6f54a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.794148 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfafd99-4ea7-4e4c-a3d8-970c0c6f54a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.794148 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfafd99-4ea7-4e4c-a3d8-970c0c6f54a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.795266 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfafd99-4ea7-4e4c-a3d8-970c0c6f54a9 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.800941 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de16fbf-dec6-499b-aeeb-913454ad51c4 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.805387 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34043922-6fc2-435a-8834-6de1d9001729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.806089 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34043922-6fc2-435a-8834-6de1d9001729 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.806359 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34043922-6fc2-435a-8834-6de1d9001729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.806635 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34043922-6fc2-435a-8834-6de1d9001729 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.807290 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34043922-6fc2-435a-8834-6de1d9001729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.808922 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34043922-6fc2-435a-8834-6de1d9001729 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.818936 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 630/1266] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:51.818936 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34043922-6fc2-435a-8834-6de1d9001729 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.824657 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:51.827263 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 637/1267] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:27:51 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:51.829953 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34043922-6fc2-435a-8834-6de1d9001729 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.831186 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfafd99-4ea7-4e4c-a3d8-970c0c6f54a9 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.887161 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c209a50-6607-4f20-a07d-b765583deea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.887881 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c209a50-6607-4f20-a07d-b765583deea4 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.888157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c209a50-6607-4f20-a07d-b765583deea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.888421 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c209a50-6607-4f20-a07d-b765583deea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.889083 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c209a50-6607-4f20-a07d-b765583deea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.890744 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c209a50-6607-4f20-a07d-b765583deea4 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.895513 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfafd99-4ea7-4e4c-a3d8-970c0c6f54a9 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.900838 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2cd6f-7d2d-4865-b625-a0bc7f69b69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.901151 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 631/1268] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:51.901458 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2cd6f-7d2d-4865-b625-a0bc7f69b69a admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.901737 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2cd6f-7d2d-4865-b625-a0bc7f69b69a admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.901954 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c209a50-6607-4f20-a07d-b765583deea4 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.902093 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2cd6f-7d2d-4865-b625-a0bc7f69b69a admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.902762 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2cd6f-7d2d-4865-b625-a0bc7f69b69a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.904513 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2cd6f-7d2d-4865-b625-a0bc7f69b69a admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.911399 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c209a50-6607-4f20-a07d-b765583deea4 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.915097 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 638/1269] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:51.916214 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2cd6f-7d2d-4865-b625-a0bc7f69b69a admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.918856 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb01c5-4944-4dd1-b36b-bbb74f4e6336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.919550 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb01c5-4944-4dd1-b36b-bbb74f4e6336 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.919824 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb01c5-4944-4dd1-b36b-bbb74f4e6336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.920093 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb01c5-4944-4dd1-b36b-bbb74f4e6336 admin admin] PATH_INFO: `/v3/projects/55270fcd0b4d4a18819be68b0e3956a3/users/7c0fcf360562419e8b114941b382fcc4/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.927793 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2cd6f-7d2d-4865-b625-a0bc7f69b69a admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.932962 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb4149e-3602-4c29-b8b9-4dde357572e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:51.933566 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4149e-3602-4c29-b8b9-4dde357572e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:51.933841 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4149e-3602-4c29-b8b9-4dde357572e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:51.934104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4149e-3602-4c29-b8b9-4dde357572e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:51.934893 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb4149e-3602-4c29-b8b9-4dde357572e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.935594 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb01c5-4944-4dd1-b36b-bbb74f4e6336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55270fcd0b4d4a18819be68b0e3956a3, user_id=7c0fcf360562419e8b114941b382fcc4, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:51.936474 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb4149e-3602-4c29-b8b9-4dde357572e4 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.938241 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb01c5-4944-4dd1-b36b-bbb74f4e6336 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:51.946626 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 639/1270] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:51.947458 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb4149e-3602-4c29-b8b9-4dde357572e4 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:51.957116 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb4149e-3602-4c29-b8b9-4dde357572e4 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:51.960631 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 632/1271] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:27:51 2022] PUT /identity/v3/projects/55270fcd0b4d4a18819be68b0e3956a3/users/7c0fcf360562419e8b114941b382fcc4/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:51.961714 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb01c5-4944-4dd1-b36b-bbb74f4e6336 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:52.018272 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e3b05-6444-4682-bee2-3b6aed150950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:52.018914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e3b05-6444-4682-bee2-3b6aed150950 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.019197 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e3b05-6444-4682-bee2-3b6aed150950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.019467 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e3b05-6444-4682-bee2-3b6aed150950 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.020105 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e3b05-6444-4682-bee2-3b6aed150950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:52.021714 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e3b05-6444-4682-bee2-3b6aed150950 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:52.025871 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb01c5-4944-4dd1-b36b-bbb74f4e6336 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:52.030714 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525b58c-e6ef-4fed-bfad-c688e8152d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:52.031291 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525b58c-e6ef-4fed-bfad-c688e8152d63 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.031589 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525b58c-e6ef-4fed-bfad-c688e8152d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.031834 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525b58c-e6ef-4fed-bfad-c688e8152d63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.032501 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1525b58c-e6ef-4fed-bfad-c688e8152d63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:52.033026 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 640/1272] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:52.034228 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1525b58c-e6ef-4fed-bfad-c688e8152d63 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:52.035641 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e3b05-6444-4682-bee2-3b6aed150950 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:52.044934 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 633/1273] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:52.045623 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e3b05-6444-4682-bee2-3b6aed150950 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:52.046644 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525b58c-e6ef-4fed-bfad-c688e8152d63 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:52.050528 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef9eb6-f708-4405-8027-b065aa5972ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:52.051264 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef9eb6-f708-4405-8027-b065aa5972ab admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.051416 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef9eb6-f708-4405-8027-b065aa5972ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.051750 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef9eb6-f708-4405-8027-b065aa5972ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.052855 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ef9eb6-f708-4405-8027-b065aa5972ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:52.055562 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ef9eb6-f708-4405-8027-b065aa5972ab admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:52.056828 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525b58c-e6ef-4fed-bfad-c688e8152d63 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:52.062866 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d89c25-f031-46f9-89e8-57b0a2a0c780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:52.063523 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d89c25-f031-46f9-89e8-57b0a2a0c780 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.063869 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d89c25-f031-46f9-89e8-57b0a2a0c780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.064116 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d89c25-f031-46f9-89e8-57b0a2a0c780 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.065608 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d89c25-f031-46f9-89e8-57b0a2a0c780 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:52.067879 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 641/1274] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:52.068023 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d89c25-f031-46f9-89e8-57b0a2a0c780 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:52.068837 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef9eb6-f708-4405-8027-b065aa5972ab admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:52.080241 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef9eb6-f708-4405-8027-b065aa5972ab admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:52.085094 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028301cd-76c0-4aa1-be74-1b07cc30aa7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:52.085907 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028301cd-76c0-4aa1-be74-1b07cc30aa7f admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.086471 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028301cd-76c0-4aa1-be74-1b07cc30aa7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.086792 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028301cd-76c0-4aa1-be74-1b07cc30aa7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.087426 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028301cd-76c0-4aa1-be74-1b07cc30aa7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:52.089079 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028301cd-76c0-4aa1-be74-1b07cc30aa7f admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:52.091599 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 634/1275] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:27:52 2022] POST /identity/v3/users => generated 299 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:52.092046 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d89c25-f031-46f9-89e8-57b0a2a0c780 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:52.100286 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 642/1276] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:52.101176 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028301cd-76c0-4aa1-be74-1b07cc30aa7f admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:52.101636 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d89c25-f031-46f9-89e8-57b0a2a0c780 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:52.106524 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:52.106649 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549513d-46fa-4c19-910b-ef77400a7f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:52.107213 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549513d-46fa-4c19-910b-ef77400a7f07 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.107492 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549513d-46fa-4c19-910b-ef77400a7f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.107761 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549513d-46fa-4c19-910b-ef77400a7f07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.108533 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2549513d-46fa-4c19-910b-ef77400a7f07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:52.110353 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2549513d-46fa-4c19-910b-ef77400a7f07 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:52.112325 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028301cd-76c0-4aa1-be74-1b07cc30aa7f admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:52.117540 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4237aa-39a2-4fe0-b646-5e57bfbe55b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:52.118211 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4237aa-39a2-4fe0-b646-5e57bfbe55b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.118487 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4237aa-39a2-4fe0-b646-5e57bfbe55b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.118785 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4237aa-39a2-4fe0-b646-5e57bfbe55b1 admin admin] PATH_INFO: `/v3/projects/9de64ee9457449bea0deedc51fd538f7/users/d568635a644d444096997355ceb21202/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.122275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 635/1277] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:52.126825 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc0cf3-a532-4eb7-af81-12abf362802d None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.127231 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc0cf3-a532-4eb7-af81-12abf362802d None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.127500 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc0cf3-a532-4eb7-af81-12abf362802d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.135540 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4237aa-39a2-4fe0-b646-5e57bfbe55b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9de64ee9457449bea0deedc51fd538f7, user_id=d568635a644d444096997355ceb21202, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:52.138210 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4237aa-39a2-4fe0-b646-5e57bfbe55b1 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:52.152524 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 643/1278] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:27:52 2022] PUT /identity/v3/projects/9de64ee9457449bea0deedc51fd538f7/users/d568635a644d444096997355ceb21202/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:52.153380 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4237aa-39a2-4fe0-b646-5e57bfbe55b1 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:52.155327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-c1cc0cf3-a532-4eb7-af81-12abf362802d None None] MFA Rules not processed for user `7c0fcf360562419e8b114941b382fcc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:52.156568 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-c1cc0cf3-a532-4eb7-af81-12abf362802d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:52.187150 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:52.215323 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4237aa-39a2-4fe0-b646-5e57bfbe55b1 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:52.219934 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc271087-b2d0-49b7-958f-98e6b8dd6b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:52.220613 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc271087-b2d0-49b7-958f-98e6b8dd6b9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.220889 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc271087-b2d0-49b7-958f-98e6b8dd6b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.221160 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc271087-b2d0-49b7-958f-98e6b8dd6b9c admin admin] PATH_INFO: `/v3/projects/296a6df3ecbd41b3a420d6f3e0ae89bb/users/338c9b8489ab45a9aabe1f518f0870d2/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.223773 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 636/1279] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:27:52 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:52.224573 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc0cf3-a532-4eb7-af81-12abf362802d None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:52.236598 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc271087-b2d0-49b7-958f-98e6b8dd6b9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=296a6df3ecbd41b3a420d6f3e0ae89bb, user_id=338c9b8489ab45a9aabe1f518f0870d2, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:52.237157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc0cf3-a532-4eb7-af81-12abf362802d None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:52.239382 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc271087-b2d0-49b7-958f-98e6b8dd6b9c admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:52.242104 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab71f55-af06-4f1f-9ead-7b86f7b090f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:52.242799 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab71f55-af06-4f1f-9ead-7b86f7b090f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.243132 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab71f55-af06-4f1f-9ead-7b86f7b090f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.243426 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab71f55-af06-4f1f-9ead-7b86f7b090f3 admin admin] PATH_INFO: `/v3/projects/9a9f191b424a43e4bc897e33be76e68e/users/16816b3849604454b4b277158f5cfa65/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.253654 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 644/1280] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:27:52 2022] PUT /identity/v3/projects/296a6df3ecbd41b3a420d6f3e0ae89bb/users/338c9b8489ab45a9aabe1f518f0870d2/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:52.254442 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc271087-b2d0-49b7-958f-98e6b8dd6b9c admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:52.265058 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab71f55-af06-4f1f-9ead-7b86f7b090f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a9f191b424a43e4bc897e33be76e68e, user_id=16816b3849604454b4b277158f5cfa65, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:52.267772 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab71f55-af06-4f1f-9ead-7b86f7b090f3 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:52.282381 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 637/1281] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:27:52 2022] PUT /identity/v3/projects/9a9f191b424a43e4bc897e33be76e68e/users/16816b3849604454b4b277158f5cfa65/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:52.286191 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab71f55-af06-4f1f-9ead-7b86f7b090f3 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:52.325749 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc271087-b2d0-49b7-958f-98e6b8dd6b9c admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:52.343776 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab71f55-af06-4f1f-9ead-7b86f7b090f3 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:52.348522 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c7e90-b45f-404e-b784-31b1bfe3b03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:52.349122 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c7e90-b45f-404e-b784-31b1bfe3b03c admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.349441 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c7e90-b45f-404e-b784-31b1bfe3b03c admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.349675 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c7e90-b45f-404e-b784-31b1bfe3b03c admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.350344 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c7e90-b45f-404e-b784-31b1bfe3b03c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:52.351972 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c7e90-b45f-404e-b784-31b1bfe3b03c admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:52.361987 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 638/1282] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:52.362938 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c7e90-b45f-404e-b784-31b1bfe3b03c admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:52.372396 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c7e90-b45f-404e-b784-31b1bfe3b03c admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:52.377034 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1abea70-732b-477a-ab06-2925748679ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:52.377631 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abea70-732b-477a-ab06-2925748679ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.378386 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abea70-732b-477a-ab06-2925748679ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.378691 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abea70-732b-477a-ab06-2925748679ca admin admin] PATH_INFO: `/v3/projects/b7c9e9f986054f9fbe25e4fe6f4b700a/users/a0098b52d51c4285be0a77dbe7d3b6a4/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.387217 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8443a0cd-c536-4d36-b521-c79e000a0bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:52.387888 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443a0cd-c536-4d36-b521-c79e000a0bd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.388614 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443a0cd-c536-4d36-b521-c79e000a0bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.388892 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443a0cd-c536-4d36-b521-c79e000a0bd1 admin admin] PATH_INFO: `/v3/projects/4fc2ce1f15e34c7491c2bdcbfc26c7f0/users/be586640013949589d2a7b11a7a2f3b1/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.395200 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1abea70-732b-477a-ab06-2925748679ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7c9e9f986054f9fbe25e4fe6f4b700a, user_id=a0098b52d51c4285be0a77dbe7d3b6a4, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:52.398286 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1abea70-732b-477a-ab06-2925748679ca admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:52.402334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:52.406398 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8443a0cd-c536-4d36-b521-c79e000a0bd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fc2ce1f15e34c7491c2bdcbfc26c7f0, user_id=be586640013949589d2a7b11a7a2f3b1, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:52.409671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8443a0cd-c536-4d36-b521-c79e000a0bd1 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:52.416652 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 639/1283] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:27:52 2022] PUT /identity/v3/projects/b7c9e9f986054f9fbe25e4fe6f4b700a/users/a0098b52d51c4285be0a77dbe7d3b6a4/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:52.417455 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1abea70-732b-477a-ab06-2925748679ca admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:52.425535 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 645/1284] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:27:52 2022] PUT /identity/v3/projects/4fc2ce1f15e34c7491c2bdcbfc26c7f0/users/be586640013949589d2a7b11a7a2f3b1/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:52.429461 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed422d79-f192-4bb7-871d-a7d9e2bdb037 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.429461 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed422d79-f192-4bb7-871d-a7d9e2bdb037 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.429461 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed422d79-f192-4bb7-871d-a7d9e2bdb037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.446401 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-ed422d79-f192-4bb7-871d-a7d9e2bdb037 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:52.447667 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-ed422d79-f192-4bb7-871d-a7d9e2bdb037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:52.483059 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1abea70-732b-477a-ab06-2925748679ca admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:52.506272 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 646/1285] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:52.506874 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:52.507102 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed422d79-f192-4bb7-871d-a7d9e2bdb037 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:52.516599 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed422d79-f192-4bb7-871d-a7d9e2bdb037 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:52.521720 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d701cb1-3825-4c36-bf1c-a8917ee9ea3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:52.522379 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d701cb1-3825-4c36-bf1c-a8917ee9ea3b admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.522641 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d701cb1-3825-4c36-bf1c-a8917ee9ea3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.522938 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d701cb1-3825-4c36-bf1c-a8917ee9ea3b admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.523565 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d701cb1-3825-4c36-bf1c-a8917ee9ea3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:52.525600 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d701cb1-3825-4c36-bf1c-a8917ee9ea3b admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:52.543572 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e095ec8-5446-433e-ac71-1078de0b52a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:52.544355 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e095ec8-5446-433e-ac71-1078de0b52a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.544576 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e095ec8-5446-433e-ac71-1078de0b52a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.544843 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e095ec8-5446-433e-ac71-1078de0b52a0 admin admin] PATH_INFO: `/v3/projects/1dfbab8dc316469880ccb755f4d91a32/users/e4f9bb9b60a246f3a84a4d4618c3f527/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.560348 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 647/1286] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:27:52 2022] POST /identity/v3/projects => generated 405 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:52.560348 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966d2f0-45c6-4f8e-a205-dd33b9adf637 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.560348 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966d2f0-45c6-4f8e-a205-dd33b9adf637 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.560726 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e095ec8-5446-433e-ac71-1078de0b52a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dfbab8dc316469880ccb755f4d91a32, user_id=e4f9bb9b60a246f3a84a4d4618c3f527, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:52.560822 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966d2f0-45c6-4f8e-a205-dd33b9adf637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.562478 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e095ec8-5446-433e-ac71-1078de0b52a0 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:52.577500 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 640/1287] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:27:52 2022] PUT /identity/v3/projects/1dfbab8dc316469880ccb755f4d91a32/users/e4f9bb9b60a246f3a84a4d4618c3f527/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:52.578836 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23978db0-aba4-465d-88c6-953e5e668e62 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.579116 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23978db0-aba4-465d-88c6-953e5e668e62 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.579406 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23978db0-aba4-465d-88c6-953e5e668e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.598746 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-a966d2f0-45c6-4f8e-a205-dd33b9adf637 None None] MFA Rules not processed for user `338c9b8489ab45a9aabe1f518f0870d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:52.599979 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-a966d2f0-45c6-4f8e-a205-dd33b9adf637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:52.604454 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-23978db0-aba4-465d-88c6-953e5e668e62 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:52.604454 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-23978db0-aba4-465d-88c6-953e5e668e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:52.667401 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 641/1288] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:52.668272 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23978db0-aba4-465d-88c6-953e5e668e62 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:52.671116 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 648/1289] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:27:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:52.676640 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18031fbb-e5ea-497e-9cde-2f76c17294df None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.676825 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18031fbb-e5ea-497e-9cde-2f76c17294df None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.677094 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18031fbb-e5ea-497e-9cde-2f76c17294df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.679143 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23978db0-aba4-465d-88c6-953e5e668e62 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:52.688320 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab190c-4426-454b-8d56-e02e8466ee6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:52.689258 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab190c-4426-454b-8d56-e02e8466ee6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.689568 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab190c-4426-454b-8d56-e02e8466ee6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.690012 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab190c-4426-454b-8d56-e02e8466ee6d admin admin] PATH_INFO: `/v3/projects/3159cb992dfb420fb3b986c8bdab05c9/users/ac9c0880124a41d886c335107a8d25ee/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.707313 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-18031fbb-e5ea-497e-9cde-2f76c17294df None None] MFA Rules not processed for user `a0098b52d51c4285be0a77dbe7d3b6a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:52.708813 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-18031fbb-e5ea-497e-9cde-2f76c17294df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:52.710227 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eab190c-4426-454b-8d56-e02e8466ee6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3159cb992dfb420fb3b986c8bdab05c9, user_id=ac9c0880124a41d886c335107a8d25ee, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:52.713750 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eab190c-4426-454b-8d56-e02e8466ee6d admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:52.727891 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 642/1290] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:27:52 2022] PUT /identity/v3/projects/3159cb992dfb420fb3b986c8bdab05c9/users/ac9c0880124a41d886c335107a8d25ee/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:52.729374 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03168982-facf-4732-919c-7c1d7201b03d None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.729652 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03168982-facf-4732-919c-7c1d7201b03d None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.729913 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03168982-facf-4732-919c-7c1d7201b03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.748239 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-03168982-facf-4732-919c-7c1d7201b03d None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:52.749427 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-03168982-facf-4732-919c-7c1d7201b03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:52.760497 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:52.780035 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 649/1291] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:27:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:52.780869 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18031fbb-e5ea-497e-9cde-2f76c17294df None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:52.811689 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 643/1292] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:52.812435 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03168982-facf-4732-919c-7c1d7201b03d None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:52.821969 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03168982-facf-4732-919c-7c1d7201b03d None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:52.826743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd98146-1acc-47d3-baf6-b4d4b6b26eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:52.827322 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd98146-1acc-47d3-baf6-b4d4b6b26eba admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.827600 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd98146-1acc-47d3-baf6-b4d4b6b26eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.827990 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd98146-1acc-47d3-baf6-b4d4b6b26eba admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.828660 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd98146-1acc-47d3-baf6-b4d4b6b26eba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:52.830566 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd98146-1acc-47d3-baf6-b4d4b6b26eba admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:52.840869 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18031fbb-e5ea-497e-9cde-2f76c17294df None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:52.852943 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 644/1293] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:27:52 2022] POST /identity/v3/users => generated 309 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:52.854155 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f8281-f0c2-4404-af70-d281325b1b9a None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.854433 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f8281-f0c2-4404-af70-d281325b1b9a None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.854723 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f8281-f0c2-4404-af70-d281325b1b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.873189 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-6c3f8281-f0c2-4404-af70-d281325b1b9a None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:52.874393 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-6c3f8281-f0c2-4404-af70-d281325b1b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:52.882444 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 645/1294] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:52.883298 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f8281-f0c2-4404-af70-d281325b1b9a None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:52.887203 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:52.895925 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c40f201-c25f-4eb1-8e02-18579a4a7816 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:52.896565 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40f201-c25f-4eb1-8e02-18579a4a7816 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.896855 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40f201-c25f-4eb1-8e02-18579a4a7816 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.897122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40f201-c25f-4eb1-8e02-18579a4a7816 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.907001 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c40f201-c25f-4eb1-8e02-18579a4a7816 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:52.908438 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c40f201-c25f-4eb1-8e02-18579a4a7816 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:52.918279 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 650/1295] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:52.919613 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce173d80-79bb-4758-a4f8-55f454facc0c None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.919896 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce173d80-79bb-4758-a4f8-55f454facc0c None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.920171 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce173d80-79bb-4758-a4f8-55f454facc0c None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.922856 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 651/1296] 10.222.0.30 () {54 vars in 961 bytes} [Wed Nov 2 07:27:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:27:52.924170 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce173d80-79bb-4758-a4f8-55f454facc0c None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:52.941641 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce173d80-79bb-4758-a4f8-55f454facc0c None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:52.948331 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f8281-f0c2-4404-af70-d281325b1b9a None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:52.948522 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de6830-7a5d-4bf3-a983-fea552af12aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:52.949055 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de6830-7a5d-4bf3-a983-fea552af12aa admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.949339 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de6830-7a5d-4bf3-a983-fea552af12aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.949624 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de6830-7a5d-4bf3-a983-fea552af12aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.950282 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de6830-7a5d-4bf3-a983-fea552af12aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:52.952415 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de6830-7a5d-4bf3-a983-fea552af12aa admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:52.953111 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9058c509-3da1-4b65-b7db-60826baafd8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:52.953930 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058c509-3da1-4b65-b7db-60826baafd8c service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:52.954406 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058c509-3da1-4b65-b7db-60826baafd8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:52.954896 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058c509-3da1-4b65-b7db-60826baafd8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:52.963557 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 652/1297] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:52.965867 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9058c509-3da1-4b65-b7db-60826baafd8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:52.967314 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de6830-7a5d-4bf3-a983-fea552af12aa admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:52.967639 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9058c509-3da1-4b65-b7db-60826baafd8c service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:52.980021 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 646/1298] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:52.981127 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9058c509-3da1-4b65-b7db-60826baafd8c service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:52.982790 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-9058c509-3da1-4b65-b7db-60826baafd8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:52.998059 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9058c509-3da1-4b65-b7db-60826baafd8c service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:53.002267 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed3706-6d1d-4015-a6fc-92aa56ed82d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:53.004142 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed3706-6d1d-4015-a6fc-92aa56ed82d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:53.004418 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed3706-6d1d-4015-a6fc-92aa56ed82d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:53.004911 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed3706-6d1d-4015-a6fc-92aa56ed82d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:53.018670 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed3706-6d1d-4015-a6fc-92aa56ed82d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:53.020468 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed3706-6d1d-4015-a6fc-92aa56ed82d3 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:53.025104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de6830-7a5d-4bf3-a983-fea552af12aa admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:53.028963 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642276e7-ecd1-4650-b821-552f9db78cef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:53.029505 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642276e7-ecd1-4650-b821-552f9db78cef service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:53.029825 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642276e7-ecd1-4650-b821-552f9db78cef service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:53.030219 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642276e7-ecd1-4650-b821-552f9db78cef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:53.032558 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 647/1299] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:53.033603 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed3706-6d1d-4015-a6fc-92aa56ed82d3 service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:53.045914 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed3706-6d1d-4015-a6fc-92aa56ed82d3 service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:53.051115 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75651e53-60ce-4032-8cea-d1ac5393ab8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:53.051711 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75651e53-60ce-4032-8cea-d1ac5393ab8b admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:53.051985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75651e53-60ce-4032-8cea-d1ac5393ab8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:53.052246 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75651e53-60ce-4032-8cea-d1ac5393ab8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:53.053152 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75651e53-60ce-4032-8cea-d1ac5393ab8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:53.055308 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75651e53-60ce-4032-8cea-d1ac5393ab8b admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:53.069181 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 648/1300] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:53.070330 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75651e53-60ce-4032-8cea-d1ac5393ab8b admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:53.086096 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75651e53-60ce-4032-8cea-d1ac5393ab8b admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:53.094111 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5de673-c7c8-4f11-b735-55e643204691 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:53.094800 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5de673-c7c8-4f11-b735-55e643204691 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:53.095170 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5de673-c7c8-4f11-b735-55e643204691 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:53.095502 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5de673-c7c8-4f11-b735-55e643204691 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:53.097787 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:53.111037 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642276e7-ecd1-4650-b821-552f9db78cef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:53.112371 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5de673-c7c8-4f11-b735-55e643204691 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:53.113075 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642276e7-ecd1-4650-b821-552f9db78cef service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:53.114074 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5de673-c7c8-4f11-b735-55e643204691 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:53.126531 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 653/1301] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:53.127516 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 649/1302] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:53.128828 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5de673-c7c8-4f11-b735-55e643204691 service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:53.129194 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868dbbdb-1d4a-463b-af3c-829b12313107 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:53.129575 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868dbbdb-1d4a-463b-af3c-829b12313107 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:53.129852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868dbbdb-1d4a-463b-af3c-829b12313107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:53.141177 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5de673-c7c8-4f11-b735-55e643204691 service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:53.148103 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7075ac-9df0-454d-8af7-d1667a1dc103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:53.150529 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7075ac-9df0-454d-8af7-d1667a1dc103 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:53.151242 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7075ac-9df0-454d-8af7-d1667a1dc103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:53.151721 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7075ac-9df0-454d-8af7-d1667a1dc103 admin admin] PATH_INFO: `/v3/projects/3159cb992dfb420fb3b986c8bdab05c9/users/ac9c0880124a41d886c335107a8d25ee/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:53.155834 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7075ac-9df0-454d-8af7-d1667a1dc103 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3159cb992dfb420fb3b986c8bdab05c9, user_id=ac9c0880124a41d886c335107a8d25ee, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:53.161356 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7075ac-9df0-454d-8af7-d1667a1dc103 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:53.177951 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 650/1303] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:27:53 2022] PUT /identity/v3/projects/3159cb992dfb420fb3b986c8bdab05c9/users/ac9c0880124a41d886c335107a8d25ee/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:53.181302 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7075ac-9df0-454d-8af7-d1667a1dc103 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:53.185401 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-868dbbdb-1d4a-463b-af3c-829b12313107 None None] MFA Rules not processed for user `e6495385dccc4fa48c05b1feb957846c`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:53.186820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-868dbbdb-1d4a-463b-af3c-829b12313107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:53.295061 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 654/1304] 10.222.0.30 () {58 vars in 1058 bytes} [Wed Nov 2 07:27:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:53.296790 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0668922-1508-4886-b2d8-65f9fb91367a None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:53.297237 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0668922-1508-4886-b2d8-65f9fb91367a None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:53.300300 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0668922-1508-4886-b2d8-65f9fb91367a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:53.302705 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7075ac-9df0-454d-8af7-d1667a1dc103 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:53.315720 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09157cc6-a189-4183-8b1d-00642fe3238c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:53.316619 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09157cc6-a189-4183-8b1d-00642fe3238c admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:53.317099 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09157cc6-a189-4183-8b1d-00642fe3238c admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:53.317567 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09157cc6-a189-4183-8b1d-00642fe3238c admin admin] PATH_INFO: `/v3/projects/a3e92780d7164175af8b5e0755d9c725/users/dc041360766149458824bda6009df4a1/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:53.349233 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09157cc6-a189-4183-8b1d-00642fe3238c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3e92780d7164175af8b5e0755d9c725, user_id=dc041360766149458824bda6009df4a1, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:53.359596 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09157cc6-a189-4183-8b1d-00642fe3238c admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:53.370608 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-a0668922-1508-4886-b2d8-65f9fb91367a None None] MFA Rules not processed for user `ac9c0880124a41d886c335107a8d25ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:53.372892 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-a0668922-1508-4886-b2d8-65f9fb91367a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:53.391963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 651/1305] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:27:53 2022] PUT /identity/v3/projects/a3e92780d7164175af8b5e0755d9c725/users/dc041360766149458824bda6009df4a1/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:53.396110 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09157cc6-a189-4183-8b1d-00642fe3238c admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:53.495005 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:53.503874 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 655/1306] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:27:53 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:53.505162 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0668922-1508-4886-b2d8-65f9fb91367a None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:53.528992 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0668922-1508-4886-b2d8-65f9fb91367a None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:53.548383 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09157cc6-a189-4183-8b1d-00642fe3238c admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:53.557836 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64544f4-2720-479b-a2fe-e82f378fef0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:53.558926 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64544f4-2720-479b-a2fe-e82f378fef0a service nova] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:53.559553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64544f4-2720-479b-a2fe-e82f378fef0a service nova] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:53.560014 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64544f4-2720-479b-a2fe-e82f378fef0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:53.660011 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:53.677789 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b33c06-acad-47f4-b95c-48de3683b9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:53.678702 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b33c06-acad-47f4-b95c-48de3683b9ee admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:53.682935 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b33c06-acad-47f4-b95c-48de3683b9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:53.683329 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b33c06-acad-47f4-b95c-48de3683b9ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:53.684090 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b33c06-acad-47f4-b95c-48de3683b9ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:53.685834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b33c06-acad-47f4-b95c-48de3683b9ee admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:53.693699 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64544f4-2720-479b-a2fe-e82f378fef0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:53.695425 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64544f4-2720-479b-a2fe-e82f378fef0a service nova] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:53.712516 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 656/1307] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:53.713575 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b33c06-acad-47f4-b95c-48de3683b9ee admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:53.721213 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 652/1308] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:53.722905 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64544f4-2720-479b-a2fe-e82f378fef0a service nova] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:53.749304 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64544f4-2720-479b-a2fe-e82f378fef0a service nova] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:53.765297 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b348bed-975e-4bcc-b467-ac6f619ad2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:53.766112 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b348bed-975e-4bcc-b467-ac6f619ad2b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:53.766278 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b348bed-975e-4bcc-b467-ac6f619ad2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:53.766436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b348bed-975e-4bcc-b467-ac6f619ad2b4 admin admin] PATH_INFO: `/v3/projects/a3e92780d7164175af8b5e0755d9c725/users/dc041360766149458824bda6009df4a1/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:53.769749 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b348bed-975e-4bcc-b467-ac6f619ad2b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3e92780d7164175af8b5e0755d9c725, user_id=dc041360766149458824bda6009df4a1, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:53.779172 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b348bed-975e-4bcc-b467-ac6f619ad2b4 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:53.807788 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 653/1309] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:27:53 2022] PUT /identity/v3/projects/a3e92780d7164175af8b5e0755d9c725/users/dc041360766149458824bda6009df4a1/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:53.822750 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ddcbbf-73b4-4947-963f-bdc4de5b3a83 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:53.823113 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ddcbbf-73b4-4947-963f-bdc4de5b3a83 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:53.823401 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ddcbbf-73b4-4947-963f-bdc4de5b3a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:53.832240 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b33c06-acad-47f4-b95c-48de3683b9ee admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:53.894530 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-a7ddcbbf-73b4-4947-963f-bdc4de5b3a83 None None] MFA Rules not processed for user `dc041360766149458824bda6009df4a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:53.897818 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-a7ddcbbf-73b4-4947-963f-bdc4de5b3a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:53.922218 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af4670a-1af6-4189-8162-08b6c3957ce5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:53.922943 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4670a-1af6-4189-8162-08b6c3957ce5 service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:53.923212 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4670a-1af6-4189-8162-08b6c3957ce5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:53.923484 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4670a-1af6-4189-8162-08b6c3957ce5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:54.067768 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af4670a-1af6-4189-8162-08b6c3957ce5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:54.069187 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af4670a-1af6-4189-8162-08b6c3957ce5 service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:54.093461 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 657/1310] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:53 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:54.155675 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 654/1311] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:27:53 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:54.202944 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af4670a-1af6-4189-8162-08b6c3957ce5 service cinder] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:54.214963 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af4670a-1af6-4189-8162-08b6c3957ce5 service cinder] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:54.228527 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce526a-45af-4fd8-97c2-42d52bbabcd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:54.229198 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce526a-45af-4fd8-97c2-42d52bbabcd8 service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:54.229471 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce526a-45af-4fd8-97c2-42d52bbabcd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:54.229784 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce526a-45af-4fd8-97c2-42d52bbabcd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:54.242628 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce526a-45af-4fd8-97c2-42d52bbabcd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:54.245889 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce526a-45af-4fd8-97c2-42d52bbabcd8 service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:54.268870 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 658/1312] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:54 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:54.532643 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ddcbbf-73b4-4947-963f-bdc4de5b3a83 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:54.606970 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e3e3b-25db-41d7-b428-0e3c8fe92dc4 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:54.610990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e3e3b-25db-41d7-b428-0e3c8fe92dc4 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:54.611610 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e3e3b-25db-41d7-b428-0e3c8fe92dc4 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:54.614089 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 659/1313] 10.222.0.30 () {54 vars in 961 bytes} [Wed Nov 2 07:27:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:27:54.620029 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120a8a6-8b3e-434d-b958-223d134a436b None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:54.620328 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120a8a6-8b3e-434d-b958-223d134a436b None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:54.620569 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120a8a6-8b3e-434d-b958-223d134a436b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:54.640062 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:54.675694 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-5120a8a6-8b3e-434d-b958-223d134a436b None None] MFA Rules not processed for user `e6495385dccc4fa48c05b1feb957846c`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:54.676630 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-5120a8a6-8b3e-434d-b958-223d134a436b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:54.683997 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ddcbbf-73b4-4947-963f-bdc4de5b3a83 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:54.692536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ebfd2f-815a-41fc-bde4-bfcaa582e316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:54.693142 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ebfd2f-815a-41fc-bde4-bfcaa582e316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:54.693777 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ebfd2f-815a-41fc-bde4-bfcaa582e316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:54.693777 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ebfd2f-815a-41fc-bde4-bfcaa582e316 admin admin] PATH_INFO: `/v3/users/53ce525fe4e04ea1a27fce3227ef7e40` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:54.695087 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ebfd2f-815a-41fc-bde4-bfcaa582e316 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53ce525fe4e04ea1a27fce3227ef7e40)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:54.698345 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ebfd2f-815a-41fc-bde4-bfcaa582e316 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:54.744965 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:54.775660 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a5ebfd2f-815a-41fc-bde4-bfcaa582e316 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53ce525fe4e04ea1a27fce3227ef7e40'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:54.776394 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a5ebfd2f-815a-41fc-bde4-bfcaa582e316 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53ce525fe4e04ea1a27fce3227ef7e40'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:54.800582 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a5ebfd2f-815a-41fc-bde4-bfcaa582e316 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53ce525fe4e04ea1a27fce3227ef7e40'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:54.804457 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 660/1314] 10.222.0.30 () {58 vars in 1058 bytes} [Wed Nov 2 07:27:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:54.809719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120a8a6-8b3e-434d-b958-223d134a436b None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:54.811172 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-5120a8a6-8b3e-434d-b958-223d134a436b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:54.827353 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120a8a6-8b3e-434d-b958-223d134a436b None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:54.833418 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424cbdf-3346-4854-9a51-0c538f75f847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:54.833903 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424cbdf-3346-4854-9a51-0c538f75f847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:54.834251 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424cbdf-3346-4854-9a51-0c538f75f847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:54.834530 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424cbdf-3346-4854-9a51-0c538f75f847 admin admin] PATH_INFO: `/v3/users/7c0fcf360562419e8b114941b382fcc4` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:54.836191 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9424cbdf-3346-4854-9a51-0c538f75f847 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c0fcf360562419e8b114941b382fcc4)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:54.837641 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: WARNING py.warnings [None req-9424cbdf-3346-4854-9a51-0c538f75f847 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:27:54.837641 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: warnings.warn(msg) Nov 02 07:27:54.837641 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: Nov 02 07:27:54.839241 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9424cbdf-3346-4854-9a51-0c538f75f847 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:54.845213 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a5ebfd2f-815a-41fc-bde4-bfcaa582e316 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53ce525fe4e04ea1a27fce3227ef7e40'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:54.863040 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a5ebfd2f-815a-41fc-bde4-bfcaa582e316 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53ce525fe4e04ea1a27fce3227ef7e40'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:54.873977 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 655/1315] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:27:54 2022] DELETE /identity/v3/users/53ce525fe4e04ea1a27fce3227ef7e40 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:54.877525 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41258d8c-5646-44cb-8fa7-2cc2c669594d None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:54.878164 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41258d8c-5646-44cb-8fa7-2cc2c669594d None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:54.878436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41258d8c-5646-44cb-8fa7-2cc2c669594d None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:54.879770 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 656/1316] 10.222.0.30 () {54 vars in 912 bytes} [Wed Nov 2 07:27:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:27:54.881312 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f67fec-78bc-4395-9b01-c0ebe36c8b4c None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:54.881642 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f67fec-78bc-4395-9b01-c0ebe36c8b4c None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:54.881919 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f67fec-78bc-4395-9b01-c0ebe36c8b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:54.914631 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-f6f67fec-78bc-4395-9b01-c0ebe36c8b4c None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:54.917042 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-f6f67fec-78bc-4395-9b01-c0ebe36c8b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:54.954451 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-9424cbdf-3346-4854-9a51-0c538f75f847 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c0fcf360562419e8b114941b382fcc4'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:54.969999 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-9424cbdf-3346-4854-9a51-0c538f75f847 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c0fcf360562419e8b114941b382fcc4'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:54.971148 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-9424cbdf-3346-4854-9a51-0c538f75f847 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c0fcf360562419e8b114941b382fcc4'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:54.995578 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-9424cbdf-3346-4854-9a51-0c538f75f847 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c0fcf360562419e8b114941b382fcc4'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:54.999628 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 657/1317] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:55.001039 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03dfb1b-51e8-49b6-bfb3-b4f1d88f88da None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:55.001385 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03dfb1b-51e8-49b6-bfb3-b4f1d88f88da None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:55.001860 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03dfb1b-51e8-49b6-bfb3-b4f1d88f88da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:55.021791 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-9424cbdf-3346-4854-9a51-0c538f75f847 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c0fcf360562419e8b114941b382fcc4'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:55.030425 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-c03dfb1b-51e8-49b6-bfb3-b4f1d88f88da None None] MFA Rules not processed for user `e6495385dccc4fa48c05b1feb957846c`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:55.031754 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-c03dfb1b-51e8-49b6-bfb3-b4f1d88f88da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:55.032011 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 661/1318] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:27:54 2022] DELETE /identity/v3/users/7c0fcf360562419e8b114941b382fcc4 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:55.033134 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424cbdf-3346-4854-9a51-0c538f75f847 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:55.037440 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-9424cbdf-3346-4854-9a51-0c538f75f847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:55.175825 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424cbdf-3346-4854-9a51-0c538f75f847 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:55.188853 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b070ffd9-4ce5-497c-b51a-0345498d4dbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:55.192771 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070ffd9-4ce5-497c-b51a-0345498d4dbb service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:55.193975 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070ffd9-4ce5-497c-b51a-0345498d4dbb service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:55.194255 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070ffd9-4ce5-497c-b51a-0345498d4dbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:55.199494 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-b070ffd9-4ce5-497c-b51a-0345498d4dbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:55.213832 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:55.250779 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 658/1319] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Nov 2 07:27:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:55.252185 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519f19c-beaa-4a4f-905f-ad7e141d0b0d None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:55.252650 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519f19c-beaa-4a4f-905f-ad7e141d0b0d None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:55.255380 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519f19c-beaa-4a4f-905f-ad7e141d0b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:55.315672 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-d519f19c-beaa-4a4f-905f-ad7e141d0b0d None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:55.318691 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-d519f19c-beaa-4a4f-905f-ad7e141d0b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:55.390237 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b070ffd9-4ce5-497c-b51a-0345498d4dbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:55.407381 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b070ffd9-4ce5-497c-b51a-0345498d4dbb service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:55.418154 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 659/1320] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:55.419381 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d519f19c-beaa-4a4f-905f-ad7e141d0b0d None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:55.420847 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-d519f19c-beaa-4a4f-905f-ad7e141d0b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:55.422300 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:55.445172 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 662/1321] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:55.456292 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a23749-a9bd-40a6-9b95-57746db25687 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:55.456673 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a23749-a9bd-40a6-9b95-57746db25687 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:55.457743 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a23749-a9bd-40a6-9b95-57746db25687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:55.496172 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d519f19c-beaa-4a4f-905f-ad7e141d0b0d None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:55.499757 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6836e3-380c-48f1-91e7-09b07e98115f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:55.500452 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6836e3-380c-48f1-91e7-09b07e98115f service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:55.500948 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6836e3-380c-48f1-91e7-09b07e98115f service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:55.501311 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6836e3-380c-48f1-91e7-09b07e98115f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:55.514125 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6836e3-380c-48f1-91e7-09b07e98115f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:55.515965 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6836e3-380c-48f1-91e7-09b07e98115f service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:55.529774 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 660/1322] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:55.530871 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6836e3-380c-48f1-91e7-09b07e98115f service cinder] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:55.532528 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-8b6836e3-380c-48f1-91e7-09b07e98115f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:55.545955 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6836e3-380c-48f1-91e7-09b07e98115f service cinder] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:55.549918 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a22cfd-ecaf-4747-8db4-b1f28f743ba2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:55.550612 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a22cfd-ecaf-4747-8db4-b1f28f743ba2 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:55.551572 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a22cfd-ecaf-4747-8db4-b1f28f743ba2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:55.551572 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a22cfd-ecaf-4747-8db4-b1f28f743ba2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:55.559747 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-e3a23749-a9bd-40a6-9b95-57746db25687 None None] MFA Rules not processed for user `16816b3849604454b4b277158f5cfa65`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:55.561671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-e3a23749-a9bd-40a6-9b95-57746db25687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:55.563140 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a22cfd-ecaf-4747-8db4-b1f28f743ba2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:55.564969 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a22cfd-ecaf-4747-8db4-b1f28f743ba2 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:55.576598 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 661/1323] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:55.578545 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e24b10-a923-4448-b971-8a0a5a319418 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:55.578906 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e24b10-a923-4448-b971-8a0a5a319418 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:55.579375 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e24b10-a923-4448-b971-8a0a5a319418 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:55.581053 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 662/1324] 10.222.0.30 () {54 vars in 912 bytes} [Wed Nov 2 07:27:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:27:55.589727 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc7758-d630-459b-be94-0220a43f5c9b None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:55.591189 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc7758-d630-459b-be94-0220a43f5c9b None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:55.591538 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc7758-d630-459b-be94-0220a43f5c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:55.644258 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-dcdc7758-d630-459b-be94-0220a43f5c9b None None] MFA Rules not processed for user `e6495385dccc4fa48c05b1feb957846c`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:55.647520 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-dcdc7758-d630-459b-be94-0220a43f5c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:55.675535 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 663/1325] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Nov 2 07:27:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:55.692485 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc7758-d630-459b-be94-0220a43f5c9b None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:55.706005 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 663/1326] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:27:55 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:55.707973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a23749-a9bd-40a6-9b95-57746db25687 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:55.709067 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc7758-d630-459b-be94-0220a43f5c9b None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:55.709688 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-e3a23749-a9bd-40a6-9b95-57746db25687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:55.718393 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a8bb7-4e66-442f-8918-822cb12e7b36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:55.720111 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a8bb7-4e66-442f-8918-822cb12e7b36 service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:55.720399 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a8bb7-4e66-442f-8918-822cb12e7b36 service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:55.720670 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a8bb7-4e66-442f-8918-822cb12e7b36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:55.731019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a23749-a9bd-40a6-9b95-57746db25687 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:55.736013 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2113df-c0e6-44cd-a34a-d63679afed7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:55.736850 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2113df-c0e6-44cd-a34a-d63679afed7b admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:55.737219 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2113df-c0e6-44cd-a34a-d63679afed7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:55.737902 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2113df-c0e6-44cd-a34a-d63679afed7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:55.738647 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2113df-c0e6-44cd-a34a-d63679afed7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:55.741076 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2113df-c0e6-44cd-a34a-d63679afed7b admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:55.746399 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a8bb7-4e66-442f-8918-822cb12e7b36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:55.750294 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a8bb7-4e66-442f-8918-822cb12e7b36 service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:55.783610 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 664/1327] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:27:55.785197 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a8bb7-4e66-442f-8918-822cb12e7b36 service cinder] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:55.786762 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-081a8bb7-4e66-442f-8918-822cb12e7b36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:55.817599 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a8bb7-4e66-442f-8918-822cb12e7b36 service cinder] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:55.820377 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 664/1328] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:27:55 2022] POST /identity/v3/projects => generated 403 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:55.827435 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2113df-c0e6-44cd-a34a-d63679afed7b admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:55.943554 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b0bad3-2f3a-44d0-87a5-1aaeee7b1a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:55.945137 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b0bad3-2f3a-44d0-87a5-1aaeee7b1a6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:55.945641 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b0bad3-2f3a-44d0-87a5-1aaeee7b1a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:55.946503 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b0bad3-2f3a-44d0-87a5-1aaeee7b1a6e admin admin] PATH_INFO: `/v3/users/338c9b8489ab45a9aabe1f518f0870d2` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:55.950206 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b0bad3-2f3a-44d0-87a5-1aaeee7b1a6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=338c9b8489ab45a9aabe1f518f0870d2)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:55.953698 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b0bad3-2f3a-44d0-87a5-1aaeee7b1a6e admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:55.969545 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:56.020056 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2113df-c0e6-44cd-a34a-d63679afed7b admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:56.033816 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd127f-65d9-4ae3-8f20-4278744765a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:56.035886 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd127f-65d9-4ae3-8f20-4278744765a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:56.036299 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd127f-65d9-4ae3-8f20-4278744765a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:56.036609 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd127f-65d9-4ae3-8f20-4278744765a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:56.037680 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dd127f-65d9-4ae3-8f20-4278744765a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:56.051507 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dd127f-65d9-4ae3-8f20-4278744765a9 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:56.070446 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-41b0bad3-2f3a-44d0-87a5-1aaeee7b1a6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '338c9b8489ab45a9aabe1f518f0870d2'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:56.071584 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-41b0bad3-2f3a-44d0-87a5-1aaeee7b1a6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '338c9b8489ab45a9aabe1f518f0870d2'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:56.089048 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-41b0bad3-2f3a-44d0-87a5-1aaeee7b1a6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '338c9b8489ab45a9aabe1f518f0870d2'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:56.107856 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 665/1329] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:27:55 2022] POST /identity/v3/users => generated 308 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:56.109325 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd127f-65d9-4ae3-8f20-4278744765a9 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:56.112569 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-e1dd127f-65d9-4ae3-8f20-4278744765a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:56.129406 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-41b0bad3-2f3a-44d0-87a5-1aaeee7b1a6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '338c9b8489ab45a9aabe1f518f0870d2'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:56.163866 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd127f-65d9-4ae3-8f20-4278744765a9 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:56.172164 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-41b0bad3-2f3a-44d0-87a5-1aaeee7b1a6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '338c9b8489ab45a9aabe1f518f0870d2'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:56.186124 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0febdd2-0a8d-4360-bebe-b9f9162fa182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:56.204115 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0febdd2-0a8d-4360-bebe-b9f9162fa182 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:56.204115 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0febdd2-0a8d-4360-bebe-b9f9162fa182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:56.204115 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0febdd2-0a8d-4360-bebe-b9f9162fa182 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:56.204448 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 665/1330] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:27:55 2022] DELETE /identity/v3/users/338c9b8489ab45a9aabe1f518f0870d2 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:56.206332 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0febdd2-0a8d-4360-bebe-b9f9162fa182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:56.219623 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274209e-2f3c-47b2-9285-c309087d32fd None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:56.219978 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274209e-2f3c-47b2-9285-c309087d32fd None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:56.220200 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274209e-2f3c-47b2-9285-c309087d32fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:56.228279 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0febdd2-0a8d-4360-bebe-b9f9162fa182 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:56.278797 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 666/1331] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:27:56.279914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0febdd2-0a8d-4360-bebe-b9f9162fa182 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:56.293253 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-9274209e-2f3c-47b2-9285-c309087d32fd None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:56.296114 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-9274209e-2f3c-47b2-9285-c309087d32fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:56.305630 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0febdd2-0a8d-4360-bebe-b9f9162fa182 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:56.437858 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:56.460882 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 666/1332] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:56.466203 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d47693-fc07-4665-a01c-ce95ce9ed07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:56.467008 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d47693-fc07-4665-a01c-ce95ce9ed07f admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:56.467329 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d47693-fc07-4665-a01c-ce95ce9ed07f admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:56.467685 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d47693-fc07-4665-a01c-ce95ce9ed07f admin admin] PATH_INFO: `/v3/projects/6d9119f458fb430482e054cca8b1fcd4/users/2180d87d41784af8a9473ba59005ca06/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:56.488125 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d47693-fc07-4665-a01c-ce95ce9ed07f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d9119f458fb430482e054cca8b1fcd4, user_id=2180d87d41784af8a9473ba59005ca06, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:56.491306 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d47693-fc07-4665-a01c-ce95ce9ed07f admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:56.512068 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 667/1333] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:27:56 2022] PUT /identity/v3/projects/6d9119f458fb430482e054cca8b1fcd4/users/2180d87d41784af8a9473ba59005ca06/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:56.515890 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274209e-2f3c-47b2-9285-c309087d32fd None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:56.534707 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d47693-fc07-4665-a01c-ce95ce9ed07f admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:56.537286 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274209e-2f3c-47b2-9285-c309087d32fd None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:56.545244 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-f0d47693-fc07-4665-a01c-ce95ce9ed07f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:56.775281 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe595bb-d6f8-40bc-a5a3-3dcd4c9124f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:56.776499 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe595bb-d6f8-40bc-a5a3-3dcd4c9124f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:56.777385 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe595bb-d6f8-40bc-a5a3-3dcd4c9124f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:56.777740 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d47693-fc07-4665-a01c-ce95ce9ed07f admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:56.778585 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe595bb-d6f8-40bc-a5a3-3dcd4c9124f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:56.779703 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe595bb-d6f8-40bc-a5a3-3dcd4c9124f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:56.782775 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe595bb-d6f8-40bc-a5a3-3dcd4c9124f1 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:56.787956 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3911cee5-d9bc-44ce-a909-6a67e3b5eb3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:56.788337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911cee5-d9bc-44ce-a909-6a67e3b5eb3c service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:56.789924 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911cee5-d9bc-44ce-a909-6a67e3b5eb3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:56.790770 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911cee5-d9bc-44ce-a909-6a67e3b5eb3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:56.816590 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 667/1334] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:27:56.818839 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3911cee5-d9bc-44ce-a909-6a67e3b5eb3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:56.821983 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe595bb-d6f8-40bc-a5a3-3dcd4c9124f1 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:56.826339 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3911cee5-d9bc-44ce-a909-6a67e3b5eb3c service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:56.844178 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe595bb-d6f8-40bc-a5a3-3dcd4c9124f1 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:56.852785 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 668/1335] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:27:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:27:56.855427 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90410860-8b22-47b7-b149-76ed07ccdae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:56.856848 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90410860-8b22-47b7-b149-76ed07ccdae6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:56.857201 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90410860-8b22-47b7-b149-76ed07ccdae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:56.857982 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90410860-8b22-47b7-b149-76ed07ccdae6 admin admin] PATH_INFO: `/v3/projects/6d9119f458fb430482e054cca8b1fcd4/users/2180d87d41784af8a9473ba59005ca06/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:56.860261 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90410860-8b22-47b7-b149-76ed07ccdae6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d9119f458fb430482e054cca8b1fcd4, user_id=2180d87d41784af8a9473ba59005ca06, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:56.863336 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90410860-8b22-47b7-b149-76ed07ccdae6 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:56.884481 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 668/1336] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:27:56 2022] PUT /identity/v3/projects/6d9119f458fb430482e054cca8b1fcd4/users/2180d87d41784af8a9473ba59005ca06/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:57.026564 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3911cee5-d9bc-44ce-a909-6a67e3b5eb3c service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:57.028739 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-3911cee5-d9bc-44ce-a909-6a67e3b5eb3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:57.081050 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3911cee5-d9bc-44ce-a909-6a67e3b5eb3c service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:57.214612 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880cd5ca-cf51-4eac-a9a5-58d69bba4e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:57.216431 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880cd5ca-cf51-4eac-a9a5-58d69bba4e46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:57.216919 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880cd5ca-cf51-4eac-a9a5-58d69bba4e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:57.217469 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880cd5ca-cf51-4eac-a9a5-58d69bba4e46 admin admin] PATH_INFO: `/v3/projects/04d1043eb7464a8987352ab1714424fc` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:57.222762 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880cd5ca-cf51-4eac-a9a5-58d69bba4e46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04d1043eb7464a8987352ab1714424fc)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:57.225386 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880cd5ca-cf51-4eac-a9a5-58d69bba4e46 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:57.319213 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-880cd5ca-cf51-4eac-a9a5-58d69bba4e46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04d1043eb7464a8987352ab1714424fc', 'request_id': 'req-880cd5ca-cf51-4eac-a9a5-58d69bba4e46'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:57.340771 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-880cd5ca-cf51-4eac-a9a5-58d69bba4e46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04d1043eb7464a8987352ab1714424fc', 'request_id': 'req-880cd5ca-cf51-4eac-a9a5-58d69bba4e46'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:57.367034 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-880cd5ca-cf51-4eac-a9a5-58d69bba4e46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04d1043eb7464a8987352ab1714424fc', 'request_id': 'req-880cd5ca-cf51-4eac-a9a5-58d69bba4e46'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:57.385665 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:57.394940 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-880cd5ca-cf51-4eac-a9a5-58d69bba4e46 admin admin] The token cache is being invalidate because project 04d1043eb7464a8987352ab1714424fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:27:57.395646 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-880cd5ca-cf51-4eac-a9a5-58d69bba4e46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:57.396880 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-880cd5ca-cf51-4eac-a9a5-58d69bba4e46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:57.404137 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 669/1337] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:27:57 2022] DELETE /identity/v3/projects/04d1043eb7464a8987352ab1714424fc => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:57.405412 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90410860-8b22-47b7-b149-76ed07ccdae6 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:57.407844 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-90410860-8b22-47b7-b149-76ed07ccdae6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:57.423422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90410860-8b22-47b7-b149-76ed07ccdae6 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:57.434504 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:57.538160 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572fa34-a3f0-4d6c-8756-932391a65309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:57.540604 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572fa34-a3f0-4d6c-8756-932391a65309 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:57.541386 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572fa34-a3f0-4d6c-8756-932391a65309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:57.541803 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572fa34-a3f0-4d6c-8756-932391a65309 admin admin] PATH_INFO: `/v3/users/ac9c0880124a41d886c335107a8d25ee` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:57.544993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572fa34-a3f0-4d6c-8756-932391a65309 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac9c0880124a41d886c335107a8d25ee)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:57.548800 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572fa34-a3f0-4d6c-8756-932391a65309 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:57.684634 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-7572fa34-a3f0-4d6c-8756-932391a65309 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac9c0880124a41d886c335107a8d25ee'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:57.686692 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-7572fa34-a3f0-4d6c-8756-932391a65309 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac9c0880124a41d886c335107a8d25ee'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:57.700647 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-7572fa34-a3f0-4d6c-8756-932391a65309 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac9c0880124a41d886c335107a8d25ee'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:57.727174 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-7572fa34-a3f0-4d6c-8756-932391a65309 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac9c0880124a41d886c335107a8d25ee'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:57.754100 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-7572fa34-a3f0-4d6c-8756-932391a65309 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac9c0880124a41d886c335107a8d25ee'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:57.764054 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 669/1338] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:27:57 2022] DELETE /identity/v3/users/ac9c0880124a41d886c335107a8d25ee => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:27:57.879117 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880cd5ca-cf51-4eac-a9a5-58d69bba4e46 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:57.880669 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-880cd5ca-cf51-4eac-a9a5-58d69bba4e46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:57.901848 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880cd5ca-cf51-4eac-a9a5-58d69bba4e46 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:58.033219 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541d993-bc94-4481-a4bc-c6d9792e30c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:58.034083 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541d993-bc94-4481-a4bc-c6d9792e30c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:58.039674 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541d993-bc94-4481-a4bc-c6d9792e30c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:58.039674 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541d993-bc94-4481-a4bc-c6d9792e30c8 admin admin] PATH_INFO: `/v3/projects/55270fcd0b4d4a18819be68b0e3956a3` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:58.039674 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3541d993-bc94-4481-a4bc-c6d9792e30c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55270fcd0b4d4a18819be68b0e3956a3)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:58.039674 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3541d993-bc94-4481-a4bc-c6d9792e30c8 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:58.257915 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-3541d993-bc94-4481-a4bc-c6d9792e30c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55270fcd0b4d4a18819be68b0e3956a3', 'request_id': 'req-3541d993-bc94-4481-a4bc-c6d9792e30c8'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:58.284225 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-3541d993-bc94-4481-a4bc-c6d9792e30c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55270fcd0b4d4a18819be68b0e3956a3', 'request_id': 'req-3541d993-bc94-4481-a4bc-c6d9792e30c8'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:58.300709 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-3541d993-bc94-4481-a4bc-c6d9792e30c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55270fcd0b4d4a18819be68b0e3956a3', 'request_id': 'req-3541d993-bc94-4481-a4bc-c6d9792e30c8'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:58.332614 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-3541d993-bc94-4481-a4bc-c6d9792e30c8 admin admin] The token cache is being invalidate because project 55270fcd0b4d4a18819be68b0e3956a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:27:58.332919 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-3541d993-bc94-4481-a4bc-c6d9792e30c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:58.339006 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-3541d993-bc94-4481-a4bc-c6d9792e30c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:58.340915 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 670/1339] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:27:57 2022] DELETE /identity/v3/projects/55270fcd0b4d4a18819be68b0e3956a3 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:58.345426 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572fa34-a3f0-4d6c-8756-932391a65309 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:58.347252 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-7572fa34-a3f0-4d6c-8756-932391a65309 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:58.369032 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572fa34-a3f0-4d6c-8756-932391a65309 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:58.517698 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c191447a-fee6-40b7-a100-3821c5c2c670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:58.518418 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191447a-fee6-40b7-a100-3821c5c2c670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:58.518773 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191447a-fee6-40b7-a100-3821c5c2c670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:58.519069 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191447a-fee6-40b7-a100-3821c5c2c670 admin admin] PATH_INFO: `/v3/users/dc041360766149458824bda6009df4a1` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:58.521053 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c191447a-fee6-40b7-a100-3821c5c2c670 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc041360766149458824bda6009df4a1)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:58.524473 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c191447a-fee6-40b7-a100-3821c5c2c670 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:58.626905 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:58.667171 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c191447a-fee6-40b7-a100-3821c5c2c670 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc041360766149458824bda6009df4a1'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:58.669083 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c191447a-fee6-40b7-a100-3821c5c2c670 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc041360766149458824bda6009df4a1'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:58.687432 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c191447a-fee6-40b7-a100-3821c5c2c670 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc041360766149458824bda6009df4a1'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:58.730437 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c191447a-fee6-40b7-a100-3821c5c2c670 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc041360766149458824bda6009df4a1'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:58.778463 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c191447a-fee6-40b7-a100-3821c5c2c670 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc041360766149458824bda6009df4a1'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:58.793667 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 670/1340] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:27:58 2022] DELETE /identity/v3/users/dc041360766149458824bda6009df4a1 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:27:59.135502 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541d993-bc94-4481-a4bc-c6d9792e30c8 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:59.140180 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-3541d993-bc94-4481-a4bc-c6d9792e30c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:59.182939 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541d993-bc94-4481-a4bc-c6d9792e30c8 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:59.285026 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb0dc8-daae-4b00-b540-5386092a358a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:59.285749 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb0dc8-daae-4b00-b540-5386092a358a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:59.286025 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb0dc8-daae-4b00-b540-5386092a358a admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:59.286385 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb0dc8-daae-4b00-b540-5386092a358a admin admin] PATH_INFO: `/v3/projects/296a6df3ecbd41b3a420d6f3e0ae89bb` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:59.287737 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edb0dc8-daae-4b00-b540-5386092a358a admin admin] RBAC: Authorizing `identity:delete_project(project_id=296a6df3ecbd41b3a420d6f3e0ae89bb)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:59.291002 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edb0dc8-daae-4b00-b540-5386092a358a admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:59.353362 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:27:59.373507 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0edb0dc8-daae-4b00-b540-5386092a358a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '296a6df3ecbd41b3a420d6f3e0ae89bb', 'request_id': 'req-0edb0dc8-daae-4b00-b540-5386092a358a'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:59.409043 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0edb0dc8-daae-4b00-b540-5386092a358a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '296a6df3ecbd41b3a420d6f3e0ae89bb', 'request_id': 'req-0edb0dc8-daae-4b00-b540-5386092a358a'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:59.426183 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0edb0dc8-daae-4b00-b540-5386092a358a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '296a6df3ecbd41b3a420d6f3e0ae89bb', 'request_id': 'req-0edb0dc8-daae-4b00-b540-5386092a358a'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:59.453346 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0edb0dc8-daae-4b00-b540-5386092a358a admin admin] The token cache is being invalidate because project 296a6df3ecbd41b3a420d6f3e0ae89bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:27:59.455230 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0edb0dc8-daae-4b00-b540-5386092a358a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:59.456475 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0edb0dc8-daae-4b00-b540-5386092a358a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:27:59.459246 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 671/1341] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:27:59 2022] DELETE /identity/v3/projects/296a6df3ecbd41b3a420d6f3e0ae89bb => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:27:59.466029 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d9d04-30b7-44f6-8030-868d08253e5e None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:59.466600 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d9d04-30b7-44f6-8030-868d08253e5e None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:59.466600 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d9d04-30b7-44f6-8030-868d08253e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:59.512870 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-199d9d04-30b7-44f6-8030-868d08253e5e None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:59.515745 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-199d9d04-30b7-44f6-8030-868d08253e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:59.657552 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 671/1342] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:27:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:59.665761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9876107-a7b6-4528-8319-ba33e30a18b1 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:59.666125 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9876107-a7b6-4528-8319-ba33e30a18b1 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:59.666404 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9876107-a7b6-4528-8319-ba33e30a18b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:59.706537 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-b9876107-a7b6-4528-8319-ba33e30a18b1 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:27:59.710595 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-b9876107-a7b6-4528-8319-ba33e30a18b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:27:59.736326 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 672/1343] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:27:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:27:59.740379 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d9d04-30b7-44f6-8030-868d08253e5e None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:59.753274 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d9d04-30b7-44f6-8030-868d08253e5e None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:59.786696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19892c4b-78ed-452e-9aca-6d12853c9b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:59.787534 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19892c4b-78ed-452e-9aca-6d12853c9b38 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:59.787817 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19892c4b-78ed-452e-9aca-6d12853c9b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:59.788087 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19892c4b-78ed-452e-9aca-6d12853c9b38 admin admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:59.788741 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19892c4b-78ed-452e-9aca-6d12853c9b38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:27:59.789038 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19892c4b-78ed-452e-9aca-6d12853c9b38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:59.793784 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19892c4b-78ed-452e-9aca-6d12853c9b38 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:59.830447 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 672/1344] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:27:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:27:59.834611 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9876107-a7b6-4528-8319-ba33e30a18b1 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:59.854088 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9876107-a7b6-4528-8319-ba33e30a18b1 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:27:59.868017 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f78e475-0ab4-41bc-9a4c-10373ff199cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:27:59.869338 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78e475-0ab4-41bc-9a4c-10373ff199cd admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:27:59.869338 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78e475-0ab4-41bc-9a4c-10373ff199cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:27:59.869842 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78e475-0ab4-41bc-9a4c-10373ff199cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:27:59.870851 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f78e475-0ab4-41bc-9a4c-10373ff199cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:27:59.884245 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f78e475-0ab4-41bc-9a4c-10373ff199cd admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:27:59.955005 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 673/1345] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:27:59 2022] POST /identity/v3/projects => generated 375 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:27:59.959186 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19892c4b-78ed-452e-9aca-6d12853c9b38 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:27:59.970979 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19892c4b-78ed-452e-9aca-6d12853c9b38 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:00.155596 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c8ff43-37a2-478c-aa2d-9e2eb5cca869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:00.157316 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8ff43-37a2-478c-aa2d-9e2eb5cca869 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:00.159098 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8ff43-37a2-478c-aa2d-9e2eb5cca869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:00.159417 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8ff43-37a2-478c-aa2d-9e2eb5cca869 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:00.160644 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c8ff43-37a2-478c-aa2d-9e2eb5cca869 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:00.165420 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c8ff43-37a2-478c-aa2d-9e2eb5cca869 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:00.215071 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 673/1346] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:27:59 2022] POST /identity/v3/users => generated 288 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:00.219476 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f78e475-0ab4-41bc-9a4c-10373ff199cd admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:00.235625 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f78e475-0ab4-41bc-9a4c-10373ff199cd admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:00.245991 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bdf394-7785-4111-b5fa-130a931a5c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:00.247132 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bdf394-7785-4111-b5fa-130a931a5c96 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:00.247132 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bdf394-7785-4111-b5fa-130a931a5c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:00.247345 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bdf394-7785-4111-b5fa-130a931a5c96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:00.247842 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bdf394-7785-4111-b5fa-130a931a5c96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:00.249853 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bdf394-7785-4111-b5fa-130a931a5c96 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:00.274732 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 674/1347] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:00.278699 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c8ff43-37a2-478c-aa2d-9e2eb5cca869 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:00.299430 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c8ff43-37a2-478c-aa2d-9e2eb5cca869 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:00.308624 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6db5b-6f95-4b7a-a322-2870de5a31d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:00.309180 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6db5b-6f95-4b7a-a322-2870de5a31d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:00.309457 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6db5b-6f95-4b7a-a322-2870de5a31d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:00.309730 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6db5b-6f95-4b7a-a322-2870de5a31d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:00.310577 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6db5b-6f95-4b7a-a322-2870de5a31d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:00.315514 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6db5b-6f95-4b7a-a322-2870de5a31d1 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:00.343639 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 674/1348] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:00.345245 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bdf394-7785-4111-b5fa-130a931a5c96 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:00.364652 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bdf394-7785-4111-b5fa-130a931a5c96 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:00.372689 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951f203-29da-4cbb-9ab0-5761edc636c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:00.373402 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951f203-29da-4cbb-9ab0-5761edc636c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:00.373674 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951f203-29da-4cbb-9ab0-5761edc636c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:00.374131 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951f203-29da-4cbb-9ab0-5761edc636c2 admin admin] PATH_INFO: `/v3/projects/1dadb9283d68453b8987c450c6cfb79e/users/a2e2c74d31f94a498e3c2282dfeafe57/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:00.406453 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d951f203-29da-4cbb-9ab0-5761edc636c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dadb9283d68453b8987c450c6cfb79e, user_id=a2e2c74d31f94a498e3c2282dfeafe57, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:00.409299 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d951f203-29da-4cbb-9ab0-5761edc636c2 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:00.443778 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 675/1349] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:00 2022] PUT /identity/v3/projects/1dadb9283d68453b8987c450c6cfb79e/users/a2e2c74d31f94a498e3c2282dfeafe57/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:00.446497 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b4e91-1be1-4f35-b314-21970dc60250 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:00.446809 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b4e91-1be1-4f35-b314-21970dc60250 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:00.447084 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b4e91-1be1-4f35-b314-21970dc60250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:00.506079 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-743b4e91-1be1-4f35-b314-21970dc60250 None None] MFA Rules not processed for user `a2e2c74d31f94a498e3c2282dfeafe57`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:00.507376 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-743b4e91-1be1-4f35-b314-21970dc60250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:00.512268 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:00.652737 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 675/1350] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:00.657165 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951f203-29da-4cbb-9ab0-5761edc636c2 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:00.763092 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951f203-29da-4cbb-9ab0-5761edc636c2 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:00.769775 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84769d-e431-4ee1-94d4-540bf0bdba76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:00.770681 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84769d-e431-4ee1-94d4-540bf0bdba76 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:00.771356 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84769d-e431-4ee1-94d4-540bf0bdba76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:00.771956 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84769d-e431-4ee1-94d4-540bf0bdba76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:00.772764 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f84769d-e431-4ee1-94d4-540bf0bdba76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:00.775548 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f84769d-e431-4ee1-94d4-540bf0bdba76 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:00.808975 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b4e91-1be1-4f35-b314-21970dc60250 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:00.810792 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-743b4e91-1be1-4f35-b314-21970dc60250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:00.843953 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 676/1351] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:00 2022] POST /identity/v3/projects => generated 375 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:28:00.844173 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b4e91-1be1-4f35-b314-21970dc60250 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:00.849073 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84769d-e431-4ee1-94d4-540bf0bdba76 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:00.957977 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:00.991273 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84769d-e431-4ee1-94d4-540bf0bdba76 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:01.010930 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef646a-7bd2-4859-a684-45dd5ed8bfaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:01.012173 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef646a-7bd2-4859-a684-45dd5ed8bfaa admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:01.013001 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef646a-7bd2-4859-a684-45dd5ed8bfaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:01.015556 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef646a-7bd2-4859-a684-45dd5ed8bfaa admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:01.016798 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef646a-7bd2-4859-a684-45dd5ed8bfaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:01.019422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a8f15-8104-462f-8caf-0eab2ab3d961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:01.021068 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef646a-7bd2-4859-a684-45dd5ed8bfaa admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:01.021513 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a8f15-8104-462f-8caf-0eab2ab3d961 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:01.021876 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a8f15-8104-462f-8caf-0eab2ab3d961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:01.022125 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a8f15-8104-462f-8caf-0eab2ab3d961 admin admin] PATH_INFO: `/v3/projects/3159cb992dfb420fb3b986c8bdab05c9` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:01.024875 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a8f15-8104-462f-8caf-0eab2ab3d961 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3159cb992dfb420fb3b986c8bdab05c9)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:01.029389 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: WARNING py.warnings [None req-2d3a8f15-8104-462f-8caf-0eab2ab3d961 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 07:28:01.029389 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: warnings.warn(msg) Nov 02 07:28:01.029389 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: Nov 02 07:28:01.031970 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a8f15-8104-462f-8caf-0eab2ab3d961 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:01.093940 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 677/1352] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:00 2022] POST /identity/v3/users => generated 294 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:01.093940 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef646a-7bd2-4859-a684-45dd5ed8bfaa admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:01.096500 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-2aef646a-7bd2-4859-a684-45dd5ed8bfaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:01.112683 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef646a-7bd2-4859-a684-45dd5ed8bfaa admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:01.122435 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c157f1c-e94b-47a6-aac6-981183816687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:01.123348 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c157f1c-e94b-47a6-aac6-981183816687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:01.123846 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c157f1c-e94b-47a6-aac6-981183816687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:01.124263 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c157f1c-e94b-47a6-aac6-981183816687 admin admin] PATH_INFO: `/v3/projects/a3e92780d7164175af8b5e0755d9c725` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:01.127385 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c157f1c-e94b-47a6-aac6-981183816687 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3e92780d7164175af8b5e0755d9c725)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:01.129828 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c157f1c-e94b-47a6-aac6-981183816687 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:01.177034 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-2d3a8f15-8104-462f-8caf-0eab2ab3d961 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3159cb992dfb420fb3b986c8bdab05c9', 'request_id': 'req-2d3a8f15-8104-462f-8caf-0eab2ab3d961'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:01.217466 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-2d3a8f15-8104-462f-8caf-0eab2ab3d961 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3159cb992dfb420fb3b986c8bdab05c9', 'request_id': 'req-2d3a8f15-8104-462f-8caf-0eab2ab3d961'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:01.240844 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-2d3a8f15-8104-462f-8caf-0eab2ab3d961 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3159cb992dfb420fb3b986c8bdab05c9', 'request_id': 'req-2d3a8f15-8104-462f-8caf-0eab2ab3d961'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:01.252964 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-3c157f1c-e94b-47a6-aac6-981183816687 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3e92780d7164175af8b5e0755d9c725', 'request_id': 'req-3c157f1c-e94b-47a6-aac6-981183816687'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:01.274053 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-2d3a8f15-8104-462f-8caf-0eab2ab3d961 admin admin] The token cache is being invalidate because project 3159cb992dfb420fb3b986c8bdab05c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:01.275073 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-2d3a8f15-8104-462f-8caf-0eab2ab3d961 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:01.276382 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-2d3a8f15-8104-462f-8caf-0eab2ab3d961 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:01.281932 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 676/1353] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:00 2022] DELETE /identity/v3/projects/3159cb992dfb420fb3b986c8bdab05c9 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:01.285273 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-3c157f1c-e94b-47a6-aac6-981183816687 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3e92780d7164175af8b5e0755d9c725', 'request_id': 'req-3c157f1c-e94b-47a6-aac6-981183816687'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:01.286620 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ebd14-0c4c-447d-a6ed-9f677ff7086f None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:01.293409 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ebd14-0c4c-447d-a6ed-9f677ff7086f None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:01.294531 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ebd14-0c4c-447d-a6ed-9f677ff7086f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:01.303596 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-3c157f1c-e94b-47a6-aac6-981183816687 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3e92780d7164175af8b5e0755d9c725', 'request_id': 'req-3c157f1c-e94b-47a6-aac6-981183816687'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:01.333109 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-3c157f1c-e94b-47a6-aac6-981183816687 admin admin] The token cache is being invalidate because project a3e92780d7164175af8b5e0755d9c725 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:01.333406 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-3c157f1c-e94b-47a6-aac6-981183816687 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:01.334137 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-3c157f1c-e94b-47a6-aac6-981183816687 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:01.335926 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 678/1354] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:01 2022] DELETE /identity/v3/projects/a3e92780d7164175af8b5e0755d9c725 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:01.336525 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c157f1c-e94b-47a6-aac6-981183816687 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:01.338148 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-3c157f1c-e94b-47a6-aac6-981183816687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:01.395645 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-8a5ebd14-0c4c-447d-a6ed-9f677ff7086f None None] MFA Rules not processed for user `2180d87d41784af8a9473ba59005ca06`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:01.396960 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-8a5ebd14-0c4c-447d-a6ed-9f677ff7086f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:01.487640 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:01.553119 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c157f1c-e94b-47a6-aac6-981183816687 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:01.563638 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350872f-9ea5-4e78-921b-6eec6cf7c5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:01.564270 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350872f-9ea5-4e78-921b-6eec6cf7c5fe admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:01.564540 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350872f-9ea5-4e78-921b-6eec6cf7c5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:01.564805 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350872f-9ea5-4e78-921b-6eec6cf7c5fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:01.565451 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350872f-9ea5-4e78-921b-6eec6cf7c5fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:01.567183 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350872f-9ea5-4e78-921b-6eec6cf7c5fe admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:01.600759 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 679/1355] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:01.610199 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46973b-3d39-4dd2-98be-1e1e3d5bafc2 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:01.610519 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46973b-3d39-4dd2-98be-1e1e3d5bafc2 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:01.610780 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46973b-3d39-4dd2-98be-1e1e3d5bafc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:01.648068 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-2f46973b-3d39-4dd2-98be-1e1e3d5bafc2 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:01.649361 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 677/1356] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:01 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:01.650674 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-2f46973b-3d39-4dd2-98be-1e1e3d5bafc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:01.654785 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d022bf-da14-4cb0-9de3-783f586246bb None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:01.655244 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d022bf-da14-4cb0-9de3-783f586246bb None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:01.656104 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d022bf-da14-4cb0-9de3-783f586246bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:01.672087 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 680/1357] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:01.673164 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46973b-3d39-4dd2-98be-1e1e3d5bafc2 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:01.684098 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46973b-3d39-4dd2-98be-1e1e3d5bafc2 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:01.691566 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b5ca3-94d1-44c6-a0aa-242a07cd3988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:01.692806 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b5ca3-94d1-44c6-a0aa-242a07cd3988 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:01.693178 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b5ca3-94d1-44c6-a0aa-242a07cd3988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:01.693543 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b5ca3-94d1-44c6-a0aa-242a07cd3988 admin admin] PATH_INFO: `/v3/projects/d7029a72e6c14fc3bb9622f277c2b093/users/c4370cfb4d894d7fb1e96a03cbeffb46/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:01.711177 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b5ca3-94d1-44c6-a0aa-242a07cd3988 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7029a72e6c14fc3bb9622f277c2b093, user_id=c4370cfb4d894d7fb1e96a03cbeffb46, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:01.714888 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-81d022bf-da14-4cb0-9de3-783f586246bb None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:01.715261 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b5ca3-94d1-44c6-a0aa-242a07cd3988 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:01.716244 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-81d022bf-da14-4cb0-9de3-783f586246bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:01.751593 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 678/1358] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:01.751593 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e6d3a-641e-4fda-8f3e-07b695df0b18 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:01.751593 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e6d3a-641e-4fda-8f3e-07b695df0b18 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:01.751593 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e6d3a-641e-4fda-8f3e-07b695df0b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:01.752050 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 681/1359] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:01 2022] PUT /identity/v3/projects/d7029a72e6c14fc3bb9622f277c2b093/users/c4370cfb4d894d7fb1e96a03cbeffb46/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:01.752050 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b5ca3-94d1-44c6-a0aa-242a07cd3988 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:01.792088 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:01.830236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-408e6d3a-641e-4fda-8f3e-07b695df0b18 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:01.832912 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-408e6d3a-641e-4fda-8f3e-07b695df0b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:01.901709 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b5ca3-94d1-44c6-a0aa-242a07cd3988 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:01.916546 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20937d-b8bf-40f8-8a0f-3c8c388ad02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:01.918871 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20937d-b8bf-40f8-8a0f-3c8c388ad02e admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:01.919785 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20937d-b8bf-40f8-8a0f-3c8c388ad02e admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:01.919980 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20937d-b8bf-40f8-8a0f-3c8c388ad02e admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:01.922124 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d20937d-b8bf-40f8-8a0f-3c8c388ad02e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:01.928462 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d20937d-b8bf-40f8-8a0f-3c8c388ad02e admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:01.967099 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 682/1360] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:01.968883 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c9e9b-ec06-42a9-ae0d-7fbeb0e4991e None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:01.969781 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c9e9b-ec06-42a9-ae0d-7fbeb0e4991e None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:01.971297 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c9e9b-ec06-42a9-ae0d-7fbeb0e4991e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:02.033854 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 679/1361] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:02.035088 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e6d3a-641e-4fda-8f3e-07b695df0b18 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:02.059477 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-184c9e9b-ec06-42a9-ae0d-7fbeb0e4991e None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:02.065616 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-184c9e9b-ec06-42a9-ae0d-7fbeb0e4991e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:02.077418 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e6d3a-641e-4fda-8f3e-07b695df0b18 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:02.082763 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b13a9f-b68f-46b1-bc1c-faf3eb196ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:02.083492 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b13a9f-b68f-46b1-bc1c-faf3eb196ac1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:02.084483 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b13a9f-b68f-46b1-bc1c-faf3eb196ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:02.084483 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b13a9f-b68f-46b1-bc1c-faf3eb196ac1 admin admin] PATH_INFO: `/v3/projects/d7029a72e6c14fc3bb9622f277c2b093/users/c4370cfb4d894d7fb1e96a03cbeffb46/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:02.086275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b13a9f-b68f-46b1-bc1c-faf3eb196ac1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7029a72e6c14fc3bb9622f277c2b093, user_id=c4370cfb4d894d7fb1e96a03cbeffb46, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:02.091170 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b13a9f-b68f-46b1-bc1c-faf3eb196ac1 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:02.096099 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 683/1362] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:02.096099 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c9e9b-ec06-42a9-ae0d-7fbeb0e4991e None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:02.113018 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c9e9b-ec06-42a9-ae0d-7fbeb0e4991e None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:02.114537 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 680/1363] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:02 2022] PUT /identity/v3/projects/d7029a72e6c14fc3bb9622f277c2b093/users/c4370cfb4d894d7fb1e96a03cbeffb46/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:02.115727 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b13a9f-b68f-46b1-bc1c-faf3eb196ac1 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:02.234798 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b13a9f-b68f-46b1-bc1c-faf3eb196ac1 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:02.260037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042360ff-cd44-419c-a235-ce6b507bacba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:02.261653 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042360ff-cd44-419c-a235-ce6b507bacba admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:02.261931 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042360ff-cd44-419c-a235-ce6b507bacba admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:02.262199 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042360ff-cd44-419c-a235-ce6b507bacba admin admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:02.262878 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042360ff-cd44-419c-a235-ce6b507bacba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:02.263178 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042360ff-cd44-419c-a235-ce6b507bacba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:02.265458 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042360ff-cd44-419c-a235-ce6b507bacba admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:02.272637 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7386f36c-f5bf-4134-beca-18fa1ffeb39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:02.273268 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386f36c-f5bf-4134-beca-18fa1ffeb39c admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:02.273558 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386f36c-f5bf-4134-beca-18fa1ffeb39c admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:02.273958 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386f36c-f5bf-4134-beca-18fa1ffeb39c admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:02.274640 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7386f36c-f5bf-4134-beca-18fa1ffeb39c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:02.274972 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7386f36c-f5bf-4134-beca-18fa1ffeb39c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:02.284441 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7386f36c-f5bf-4134-beca-18fa1ffeb39c admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:02.321721 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 681/1364] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:28:02.321721 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa741d31-2f11-450f-bc2a-8a2c525de003 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:02.321721 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa741d31-2f11-450f-bc2a-8a2c525de003 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:02.321721 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa741d31-2f11-450f-bc2a-8a2c525de003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:02.324175 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 684/1365] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:28:02.325025 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7386f36c-f5bf-4134-beca-18fa1ffeb39c admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:02.397038 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7386f36c-f5bf-4134-beca-18fa1ffeb39c admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:02.416610 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88b7d2-c709-4635-8aa9-864981d5011b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:02.417923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88b7d2-c709-4635-8aa9-864981d5011b admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:02.418220 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88b7d2-c709-4635-8aa9-864981d5011b admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:02.418478 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88b7d2-c709-4635-8aa9-864981d5011b admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:02.419154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88b7d2-c709-4635-8aa9-864981d5011b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:02.421091 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88b7d2-c709-4635-8aa9-864981d5011b admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:02.452225 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-fa741d31-2f11-450f-bc2a-8a2c525de003 None None] MFA Rules not processed for user `c4370cfb4d894d7fb1e96a03cbeffb46`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:02.453493 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-fa741d31-2f11-450f-bc2a-8a2c525de003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:02.508384 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 685/1366] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:02 2022] POST /identity/v3/projects => generated 369 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:02.508384 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88b7d2-c709-4635-8aa9-864981d5011b admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:02.508629 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:02.608787 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 682/1367] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:02.617453 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa741d31-2f11-450f-bc2a-8a2c525de003 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:02.653302 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa741d31-2f11-450f-bc2a-8a2c525de003 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:02.669733 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574e9d9-582e-41b8-b748-c5452057af96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:02.670401 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574e9d9-582e-41b8-b748-c5452057af96 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:02.670772 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574e9d9-582e-41b8-b748-c5452057af96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:02.671200 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574e9d9-582e-41b8-b748-c5452057af96 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:02.671739 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6574e9d9-582e-41b8-b748-c5452057af96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:02.673674 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88b7d2-c709-4635-8aa9-864981d5011b admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:02.684581 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a643c9-97f6-4914-96cd-69de5a16ee50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:02.685223 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a643c9-97f6-4914-96cd-69de5a16ee50 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:02.685503 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a643c9-97f6-4914-96cd-69de5a16ee50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:02.685815 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a643c9-97f6-4914-96cd-69de5a16ee50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:02.685932 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6574e9d9-582e-41b8-b748-c5452057af96 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:02.686509 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a643c9-97f6-4914-96cd-69de5a16ee50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:02.689119 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a643c9-97f6-4914-96cd-69de5a16ee50 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:02.742163 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 683/1368] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:02 2022] POST /identity/v3/users => generated 285 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:02.743335 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574e9d9-582e-41b8-b748-c5452057af96 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:02.749652 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-6574e9d9-582e-41b8-b748-c5452057af96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:02.753939 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 686/1369] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:02 2022] POST /identity/v3/projects => generated 383 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:02.756505 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a643c9-97f6-4914-96cd-69de5a16ee50 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:02.780056 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:02.927869 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574e9d9-582e-41b8-b748-c5452057af96 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:02.931432 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a643c9-97f6-4914-96cd-69de5a16ee50 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:02.943445 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fcc7e6-fc33-4f78-bcc8-0f723a2f118b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:02.944095 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fcc7e6-fc33-4f78-bcc8-0f723a2f118b admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:02.944425 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fcc7e6-fc33-4f78-bcc8-0f723a2f118b admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:02.944704 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fcc7e6-fc33-4f78-bcc8-0f723a2f118b admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:02.945357 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fcc7e6-fc33-4f78-bcc8-0f723a2f118b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:02.958043 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fcc7e6-fc33-4f78-bcc8-0f723a2f118b admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:02.979156 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 687/1370] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:02.984776 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fcc7e6-fc33-4f78-bcc8-0f723a2f118b admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:03.010360 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fcc7e6-fc33-4f78-bcc8-0f723a2f118b admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:03.025891 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a415ef-07e6-4f79-b69b-f1a897ed927f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:03.026571 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a415ef-07e6-4f79-b69b-f1a897ed927f admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:03.026883 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a415ef-07e6-4f79-b69b-f1a897ed927f admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:03.027929 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a415ef-07e6-4f79-b69b-f1a897ed927f admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:03.028822 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a415ef-07e6-4f79-b69b-f1a897ed927f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:03.034805 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a415ef-07e6-4f79-b69b-f1a897ed927f admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:03.055593 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108bffe6-47a5-4084-b746-7cc220164387 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:03.056415 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108bffe6-47a5-4084-b746-7cc220164387 service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:03.056751 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108bffe6-47a5-4084-b746-7cc220164387 service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:03.057086 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108bffe6-47a5-4084-b746-7cc220164387 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:03.061442 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-108bffe6-47a5-4084-b746-7cc220164387 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:03.071622 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 688/1371] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:02 2022] POST /identity/v3/users => generated 292 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:03.076865 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a415ef-07e6-4f79-b69b-f1a897ed927f admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:03.129785 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a415ef-07e6-4f79-b69b-f1a897ed927f admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:03.150259 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edf48d5-6a10-49fa-9f2b-b7d2b2ec335c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:03.150575 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf48d5-6a10-49fa-9f2b-b7d2b2ec335c admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:03.150885 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf48d5-6a10-49fa-9f2b-b7d2b2ec335c admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:03.151155 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf48d5-6a10-49fa-9f2b-b7d2b2ec335c admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:03.151814 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edf48d5-6a10-49fa-9f2b-b7d2b2ec335c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:03.153436 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edf48d5-6a10-49fa-9f2b-b7d2b2ec335c admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:03.183708 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 689/1372] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:03.189265 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edf48d5-6a10-49fa-9f2b-b7d2b2ec335c admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:03.210548 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edf48d5-6a10-49fa-9f2b-b7d2b2ec335c admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:03.226462 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266d057-8ef9-41dd-971b-f784a7a4babb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:03.227189 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266d057-8ef9-41dd-971b-f784a7a4babb admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:03.227472 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266d057-8ef9-41dd-971b-f784a7a4babb admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:03.227744 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266d057-8ef9-41dd-971b-f784a7a4babb admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:03.228522 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7266d057-8ef9-41dd-971b-f784a7a4babb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:03.231257 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7266d057-8ef9-41dd-971b-f784a7a4babb admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:03.255950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 690/1373] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:03.256268 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108bffe6-47a5-4084-b746-7cc220164387 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:03.256841 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266d057-8ef9-41dd-971b-f784a7a4babb admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:03.259275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108bffe6-47a5-4084-b746-7cc220164387 service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:03.279249 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 684/1374] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:03.283770 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108bffe6-47a5-4084-b746-7cc220164387 service cinder] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:03.297338 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266d057-8ef9-41dd-971b-f784a7a4babb admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:03.310711 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108bffe6-47a5-4084-b746-7cc220164387 service cinder] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:03.314402 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e6ebac-14ce-4dcf-8445-11cd15470ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:03.315698 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6ebac-14ce-4dcf-8445-11cd15470ed9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:03.316184 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6ebac-14ce-4dcf-8445-11cd15470ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:03.316695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6ebac-14ce-4dcf-8445-11cd15470ed9 admin admin] PATH_INFO: `/v3/projects/8e238188c8004e3fb7d2cc03d0da9c85/users/3b1de10cc5c94e1e867d01aa34f2647d/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:03.324324 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199e1c91-dc52-400e-b4fb-1c454d8ebc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:03.324957 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e1c91-dc52-400e-b4fb-1c454d8ebc89 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:03.325326 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e1c91-dc52-400e-b4fb-1c454d8ebc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:03.325598 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e1c91-dc52-400e-b4fb-1c454d8ebc89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:03.326550 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199e1c91-dc52-400e-b4fb-1c454d8ebc89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:03.327954 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199e1c91-dc52-400e-b4fb-1c454d8ebc89 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:03.347543 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 685/1375] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:03.353350 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199e1c91-dc52-400e-b4fb-1c454d8ebc89 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:03.357545 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e6ebac-14ce-4dcf-8445-11cd15470ed9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e238188c8004e3fb7d2cc03d0da9c85, user_id=3b1de10cc5c94e1e867d01aa34f2647d, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:03.360210 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e6ebac-14ce-4dcf-8445-11cd15470ed9 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:03.363110 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:03.376114 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199e1c91-dc52-400e-b4fb-1c454d8ebc89 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:03.398709 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 691/1376] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:03 2022] PUT /identity/v3/projects/8e238188c8004e3fb7d2cc03d0da9c85/users/3b1de10cc5c94e1e867d01aa34f2647d/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:03.404728 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294534df-9361-4054-aaed-b5f99316e62b None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:03.405021 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294534df-9361-4054-aaed-b5f99316e62b None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:03.405196 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294534df-9361-4054-aaed-b5f99316e62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:03.406523 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43df975-2214-43fa-8c12-114eafa85a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:03.407233 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43df975-2214-43fa-8c12-114eafa85a61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:03.407517 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43df975-2214-43fa-8c12-114eafa85a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:03.407826 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43df975-2214-43fa-8c12-114eafa85a61 admin admin] PATH_INFO: `/v3/projects/54199d35d6944698be0cb633ad56c7e5/users/3f3fd7a3a86c48d1925d00edcb2caec2/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:03.447935 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43df975-2214-43fa-8c12-114eafa85a61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54199d35d6944698be0cb633ad56c7e5, user_id=3f3fd7a3a86c48d1925d00edcb2caec2, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:03.450737 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43df975-2214-43fa-8c12-114eafa85a61 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:03.462085 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-294534df-9361-4054-aaed-b5f99316e62b None None] MFA Rules not processed for user `3b1de10cc5c94e1e867d01aa34f2647d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:03.463968 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-294534df-9361-4054-aaed-b5f99316e62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:03.476710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 686/1377] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:03 2022] PUT /identity/v3/projects/54199d35d6944698be0cb633ad56c7e5/users/3f3fd7a3a86c48d1925d00edcb2caec2/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:03.487579 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298dc064-87a0-462f-bdcd-79358196c95e None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:03.487861 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298dc064-87a0-462f-bdcd-79358196c95e None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:03.488132 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298dc064-87a0-462f-bdcd-79358196c95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:03.545035 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-298dc064-87a0-462f-bdcd-79358196c95e None None] MFA Rules not processed for user `3f3fd7a3a86c48d1925d00edcb2caec2`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:03.550170 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-298dc064-87a0-462f-bdcd-79358196c95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:03.642079 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 692/1378] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:03.644080 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294534df-9361-4054-aaed-b5f99316e62b None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:03.719895 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 687/1379] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:03.725785 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:03.733957 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298dc064-87a0-462f-bdcd-79358196c95e None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:03.735754 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-298dc064-87a0-462f-bdcd-79358196c95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:03.816597 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294534df-9361-4054-aaed-b5f99316e62b None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:03.911320 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298dc064-87a0-462f-bdcd-79358196c95e None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:03.925966 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323e716-587e-462d-b7d1-eb54318c717f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:03.926602 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323e716-587e-462d-b7d1-eb54318c717f service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:03.926923 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323e716-587e-462d-b7d1-eb54318c717f service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:03.927172 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323e716-587e-462d-b7d1-eb54318c717f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:03.947830 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9323e716-587e-462d-b7d1-eb54318c717f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:03.950896 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9323e716-587e-462d-b7d1-eb54318c717f service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:03.964909 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 688/1380] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:28:04.039684 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cffc65-ca37-4671-97b7-df3b1c4c9a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:04.040658 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cffc65-ca37-4671-97b7-df3b1c4c9a8c admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:04.041064 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cffc65-ca37-4671-97b7-df3b1c4c9a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:04.042005 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cffc65-ca37-4671-97b7-df3b1c4c9a8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:04.042182 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cffc65-ca37-4671-97b7-df3b1c4c9a8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:04.044215 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cffc65-ca37-4671-97b7-df3b1c4c9a8c admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:04.112616 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 693/1381] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:03 2022] POST /identity/v3/projects => generated 371 bytes in 469 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:04.117518 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323e716-587e-462d-b7d1-eb54318c717f service cinder] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:04.141272 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323e716-587e-462d-b7d1-eb54318c717f service cinder] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:04.320291 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef71705-626f-40f6-a671-311920bdb96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:04.321101 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef71705-626f-40f6-a671-311920bdb96e admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:04.321298 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef71705-626f-40f6-a671-311920bdb96e admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:04.321573 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef71705-626f-40f6-a671-311920bdb96e admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:04.322217 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef71705-626f-40f6-a671-311920bdb96e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:04.327102 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef71705-626f-40f6-a671-311920bdb96e admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:04.388201 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:04.408549 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cffc65-ca37-4671-97b7-df3b1c4c9a8c admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:04.409639 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 689/1382] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:04 2022] POST /identity/v3/users => generated 292 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:28:04.428257 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cffc65-ca37-4671-97b7-df3b1c4c9a8c admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:04.456836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cd64ec-20fd-4f7a-ad16-aed628d2b027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:04.457482 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd64ec-20fd-4f7a-ad16-aed628d2b027 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:04.457771 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd64ec-20fd-4f7a-ad16-aed628d2b027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:04.458032 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd64ec-20fd-4f7a-ad16-aed628d2b027 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:04.458777 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cd64ec-20fd-4f7a-ad16-aed628d2b027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:04.460372 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cd64ec-20fd-4f7a-ad16-aed628d2b027 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:04.474185 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e107a-a0bb-4932-a9e7-97ccc9beeb80 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:04.474474 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e107a-a0bb-4932-a9e7-97ccc9beeb80 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:04.475333 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e107a-a0bb-4932-a9e7-97ccc9beeb80 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:04.478832 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 690/1383] 10.222.0.30 () {54 vars in 921 bytes} [Wed Nov 2 07:28:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:28:04.497967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 694/1384] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:28:04.503870 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e107a-a0bb-4932-a9e7-97ccc9beeb80 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:04.520184 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6bd627-2141-47b7-9159-2c5d8b718c95 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:04.520473 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6bd627-2141-47b7-9159-2c5d8b718c95 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:04.520749 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6bd627-2141-47b7-9159-2c5d8b718c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:04.524751 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e107a-a0bb-4932-a9e7-97ccc9beeb80 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:04.538692 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca84bab-ec21-4cab-905e-51b54dd0b451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:04.540590 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca84bab-ec21-4cab-905e-51b54dd0b451 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:04.541273 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca84bab-ec21-4cab-905e-51b54dd0b451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:04.541821 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca84bab-ec21-4cab-905e-51b54dd0b451 admin admin] PATH_INFO: `/v3/projects/3a667c5fa8ee4c73ae7130e24a55ce16/users/56c8bcb0e84149f89af893a698a3d9e1/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:04.562252 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-ae6bd627-2141-47b7-9159-2c5d8b718c95 None None] MFA Rules not processed for user `932d3076fb4b426d8c83d02eb60d8063`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:04.563289 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-ae6bd627-2141-47b7-9159-2c5d8b718c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:04.564515 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca84bab-ec21-4cab-905e-51b54dd0b451 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a667c5fa8ee4c73ae7130e24a55ce16, user_id=56c8bcb0e84149f89af893a698a3d9e1, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:04.569847 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca84bab-ec21-4cab-905e-51b54dd0b451 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:04.602743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 691/1385] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:04 2022] PUT /identity/v3/projects/3a667c5fa8ee4c73ae7130e24a55ce16/users/56c8bcb0e84149f89af893a698a3d9e1/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:04.606700 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca84bab-ec21-4cab-905e-51b54dd0b451 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:04.726136 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 695/1386] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Nov 2 07:28:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:04.800451 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca84bab-ec21-4cab-905e-51b54dd0b451 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:04.823373 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cda13b-c754-4c16-87fd-09c5cf9e7ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:04.826324 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cda13b-c754-4c16-87fd-09c5cf9e7ec0 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:04.826324 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cda13b-c754-4c16-87fd-09c5cf9e7ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:04.826324 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cda13b-c754-4c16-87fd-09c5cf9e7ec0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:04.827920 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cda13b-c754-4c16-87fd-09c5cf9e7ec0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:04.829772 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cda13b-c754-4c16-87fd-09c5cf9e7ec0 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:04.848706 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 692/1387] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:28:04.853538 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6bd627-2141-47b7-9159-2c5d8b718c95 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:04.874362 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6bd627-2141-47b7-9159-2c5d8b718c95 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:04.891936 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab115c6-4685-46e5-aa06-96606154aadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:04.892837 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab115c6-4685-46e5-aa06-96606154aadb admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:04.893223 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab115c6-4685-46e5-aa06-96606154aadb admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:04.893643 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab115c6-4685-46e5-aa06-96606154aadb admin admin] PATH_INFO: `/v3/projects/3a667c5fa8ee4c73ae7130e24a55ce16/users/56c8bcb0e84149f89af893a698a3d9e1/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:04.895957 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab115c6-4685-46e5-aa06-96606154aadb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a667c5fa8ee4c73ae7130e24a55ce16, user_id=56c8bcb0e84149f89af893a698a3d9e1, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:04.898858 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab115c6-4685-46e5-aa06-96606154aadb admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:04.926552 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 696/1388] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:04 2022] PUT /identity/v3/projects/3a667c5fa8ee4c73ae7130e24a55ce16/users/56c8bcb0e84149f89af893a698a3d9e1/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:04.930306 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea8451-1ec4-4028-8aab-053dc6ac4129 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:04.930561 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea8451-1ec4-4028-8aab-053dc6ac4129 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:04.930692 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea8451-1ec4-4028-8aab-053dc6ac4129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:04.987858 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-39ea8451-1ec4-4028-8aab-053dc6ac4129 None None] MFA Rules not processed for user `56c8bcb0e84149f89af893a698a3d9e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:04.989254 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-39ea8451-1ec4-4028-8aab-053dc6ac4129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:05.116107 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 693/1389] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:04 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:05.128404 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab115c6-4685-46e5-aa06-96606154aadb admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:05.212035 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:05.264104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab115c6-4685-46e5-aa06-96606154aadb admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:05.276657 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41582cc-8980-4200-9af2-f664b0e70b9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:05.277517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41582cc-8980-4200-9af2-f664b0e70b9a service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:05.278466 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41582cc-8980-4200-9af2-f664b0e70b9a service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:05.278893 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41582cc-8980-4200-9af2-f664b0e70b9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:05.294095 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea8451-1ec4-4028-8aab-053dc6ac4129 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:05.296140 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-39ea8451-1ec4-4028-8aab-053dc6ac4129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:05.315632 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41582cc-8980-4200-9af2-f664b0e70b9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:05.329008 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41582cc-8980-4200-9af2-f664b0e70b9a service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:05.368272 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 697/1390] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:05 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:05.444400 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:05.495544 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea8451-1ec4-4028-8aab-053dc6ac4129 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:05.509861 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fcc74f-9f3d-4416-bad1-3cbe73958d84 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932d3076fb4b426d8c83d02eb60d8063', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:05.510504 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcc74f-9f3d-4416-bad1-3cbe73958d84 service placement] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:05.510804 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcc74f-9f3d-4416-bad1-3cbe73958d84 service placement] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:05.511074 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcc74f-9f3d-4416-bad1-3cbe73958d84 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:05.532723 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fcc74f-9f3d-4416-bad1-3cbe73958d84 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:05.533827 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fcc74f-9f3d-4416-bad1-3cbe73958d84 service placement] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:05.565115 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 694/1391] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:28:06.140729 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41582cc-8980-4200-9af2-f664b0e70b9a service cinder] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:06.142198 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-f41582cc-8980-4200-9af2-f664b0e70b9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:06.445401 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41582cc-8980-4200-9af2-f664b0e70b9a service cinder] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:06.451188 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28d2431-8f70-4674-9fa5-297d0a24038f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:06.452076 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d2431-8f70-4674-9fa5-297d0a24038f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:06.452519 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d2431-8f70-4674-9fa5-297d0a24038f admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:06.452933 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d2431-8f70-4674-9fa5-297d0a24038f admin admin] PATH_INFO: `/v3/users/a0098b52d51c4285be0a77dbe7d3b6a4` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:06.454297 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28d2431-8f70-4674-9fa5-297d0a24038f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0098b52d51c4285be0a77dbe7d3b6a4)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:06.456999 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28d2431-8f70-4674-9fa5-297d0a24038f admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:06.559002 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fcc74f-9f3d-4416-bad1-3cbe73958d84 service placement] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:06.594969 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fcc74f-9f3d-4416-bad1-3cbe73958d84 service placement] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:06.607228 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c28d2431-8f70-4674-9fa5-297d0a24038f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0098b52d51c4285be0a77dbe7d3b6a4'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:06.620194 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c28d2431-8f70-4674-9fa5-297d0a24038f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0098b52d51c4285be0a77dbe7d3b6a4'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:06.621230 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c28d2431-8f70-4674-9fa5-297d0a24038f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0098b52d51c4285be0a77dbe7d3b6a4'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:06.621644 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c1cd6-a0eb-4e75-ad6f-63e089013905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:06.622310 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c1cd6-a0eb-4e75-ad6f-63e089013905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:06.622592 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c1cd6-a0eb-4e75-ad6f-63e089013905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:06.622978 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c1cd6-a0eb-4e75-ad6f-63e089013905 admin admin] PATH_INFO: `/v3/users/3b1de10cc5c94e1e867d01aa34f2647d` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:06.625000 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894c1cd6-a0eb-4e75-ad6f-63e089013905 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b1de10cc5c94e1e867d01aa34f2647d)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:06.627975 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894c1cd6-a0eb-4e75-ad6f-63e089013905 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:06.665784 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c28d2431-8f70-4674-9fa5-297d0a24038f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0098b52d51c4285be0a77dbe7d3b6a4'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:06.693265 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c28d2431-8f70-4674-9fa5-297d0a24038f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0098b52d51c4285be0a77dbe7d3b6a4'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:06.740637 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 698/1392] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:06 2022] DELETE /identity/v3/users/a0098b52d51c4285be0a77dbe7d3b6a4 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:06.746378 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3ad61-19ac-462d-abfc-08e2f853b3db None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:06.746686 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3ad61-19ac-462d-abfc-08e2f853b3db None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:06.747013 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3ad61-19ac-462d-abfc-08e2f853b3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:06.834940 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-84d3ad61-19ac-462d-abfc-08e2f853b3db None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:06.843211 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-84d3ad61-19ac-462d-abfc-08e2f853b3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:06.846166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-894c1cd6-a0eb-4e75-ad6f-63e089013905 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b1de10cc5c94e1e867d01aa34f2647d'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:06.847646 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-894c1cd6-a0eb-4e75-ad6f-63e089013905 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b1de10cc5c94e1e867d01aa34f2647d'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:06.880469 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-894c1cd6-a0eb-4e75-ad6f-63e089013905 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b1de10cc5c94e1e867d01aa34f2647d'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:06.920987 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-894c1cd6-a0eb-4e75-ad6f-63e089013905 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b1de10cc5c94e1e867d01aa34f2647d'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:06.945440 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-894c1cd6-a0eb-4e75-ad6f-63e089013905 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b1de10cc5c94e1e867d01aa34f2647d'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:06.958711 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 695/1393] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:06 2022] DELETE /identity/v3/users/3b1de10cc5c94e1e867d01aa34f2647d => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:06.964191 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753a1a4-b73b-45fe-9385-850087b4e5e4 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:06.964476 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753a1a4-b73b-45fe-9385-850087b4e5e4 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:06.967291 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753a1a4-b73b-45fe-9385-850087b4e5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:07.039535 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:07.063050 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-5753a1a4-b73b-45fe-9385-850087b4e5e4 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:07.079841 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-5753a1a4-b73b-45fe-9385-850087b4e5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:07.100380 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 699/1394] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:07.101472 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57bb38-d909-41d6-a58c-3a2db2277d2b None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:07.101781 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57bb38-d909-41d6-a58c-3a2db2277d2b None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:07.102080 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57bb38-d909-41d6-a58c-3a2db2277d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:07.183495 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-9d57bb38-d909-41d6-a58c-3a2db2277d2b None None] MFA Rules not processed for user `e4f9bb9b60a246f3a84a4d4618c3f527`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:07.186320 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-9d57bb38-d909-41d6-a58c-3a2db2277d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:07.227253 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 696/1395] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:07.228690 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5753a1a4-b73b-45fe-9385-850087b4e5e4 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:07.230207 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-5753a1a4-b73b-45fe-9385-850087b4e5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:07.356969 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 700/1396] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:07 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:07.358036 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57bb38-d909-41d6-a58c-3a2db2277d2b None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:07.359498 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-9d57bb38-d909-41d6-a58c-3a2db2277d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:07.380805 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5753a1a4-b73b-45fe-9385-850087b4e5e4 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:07.388293 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d438b227-9b81-4c48-bd9d-29a4d5c2ccd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:07.389121 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438b227-9b81-4c48-bd9d-29a4d5c2ccd9 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:07.395562 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438b227-9b81-4c48-bd9d-29a4d5c2ccd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:07.396846 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57bb38-d909-41d6-a58c-3a2db2277d2b None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:07.397250 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438b227-9b81-4c48-bd9d-29a4d5c2ccd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:07.402674 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7867dc4b-202a-45f8-93d7-d9a3e89c9919 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:07.403438 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867dc4b-202a-45f8-93d7-d9a3e89c9919 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:07.403805 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867dc4b-202a-45f8-93d7-d9a3e89c9919 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:07.404163 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867dc4b-202a-45f8-93d7-d9a3e89c9919 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:07.417223 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7867dc4b-202a-45f8-93d7-d9a3e89c9919 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:07.419038 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d438b227-9b81-4c48-bd9d-29a4d5c2ccd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:07.419611 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7867dc4b-202a-45f8-93d7-d9a3e89c9919 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:07.420672 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d438b227-9b81-4c48-bd9d-29a4d5c2ccd9 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:07.433820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 701/1397] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:07.435828 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da8e52-31d7-498a-a578-52172d4ceb91 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:07.436452 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da8e52-31d7-498a-a578-52172d4ceb91 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:07.436934 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da8e52-31d7-498a-a578-52172d4ceb91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:07.449607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 697/1398] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:07.462347 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-96da8e52-31d7-498a-a578-52172d4ceb91 None None] MFA Rules not processed for user `e4f9bb9b60a246f3a84a4d4618c3f527`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:07.464608 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-96da8e52-31d7-498a-a578-52172d4ceb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:07.480368 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 702/1399] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:07 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:07.490871 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d438b227-9b81-4c48-bd9d-29a4d5c2ccd9 service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:07.492326 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-d438b227-9b81-4c48-bd9d-29a4d5c2ccd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:07.501347 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:07.613615 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d438b227-9b81-4c48-bd9d-29a4d5c2ccd9 service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:07.621530 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dbbca1-3bce-49bb-aaa7-04210c686bf4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:07.622732 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbbca1-3bce-49bb-aaa7-04210c686bf4 service nova] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:07.623161 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbbca1-3bce-49bb-aaa7-04210c686bf4 service nova] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:07.624113 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbbca1-3bce-49bb-aaa7-04210c686bf4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:07.635687 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dbbca1-3bce-49bb-aaa7-04210c686bf4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:07.637619 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dbbca1-3bce-49bb-aaa7-04210c686bf4 service nova] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:07.651166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 698/1400] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:07 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:07.737941 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaaa209-6109-4a02-a3aa-41ed2a75cd84 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:07.741875 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaaa209-6109-4a02-a3aa-41ed2a75cd84 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:07.741875 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaaa209-6109-4a02-a3aa-41ed2a75cd84 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:07.743942 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 703/1401] 10.222.0.30 () {54 vars in 921 bytes} [Wed Nov 2 07:28:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:28:07.759881 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535bf9dd-de3e-431d-8fc0-aae6cc5c66e9 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:07.760153 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535bf9dd-de3e-431d-8fc0-aae6cc5c66e9 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:07.760520 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535bf9dd-de3e-431d-8fc0-aae6cc5c66e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:07.808815 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-535bf9dd-de3e-431d-8fc0-aae6cc5c66e9 None None] MFA Rules not processed for user `932d3076fb4b426d8c83d02eb60d8063`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:07.810481 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-535bf9dd-de3e-431d-8fc0-aae6cc5c66e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:07.851854 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaaa209-6109-4a02-a3aa-41ed2a75cd84 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:07.855451 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-ecaaa209-6109-4a02-a3aa-41ed2a75cd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:07.885313 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaaa209-6109-4a02-a3aa-41ed2a75cd84 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:07.893483 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545582f8-4041-4fa8-b895-dfd96f2b4647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:07.895560 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545582f8-4041-4fa8-b895-dfd96f2b4647 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:07.896244 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545582f8-4041-4fa8-b895-dfd96f2b4647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:07.897425 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545582f8-4041-4fa8-b895-dfd96f2b4647 admin admin] PATH_INFO: `/v3/users/16816b3849604454b4b277158f5cfa65` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:07.899872 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545582f8-4041-4fa8-b895-dfd96f2b4647 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16816b3849604454b4b277158f5cfa65)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:07.904434 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545582f8-4041-4fa8-b895-dfd96f2b4647 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:07.944921 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 699/1402] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Nov 2 07:28:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:07.976731 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535bf9dd-de3e-431d-8fc0-aae6cc5c66e9 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:07.978836 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-535bf9dd-de3e-431d-8fc0-aae6cc5c66e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:08.007306 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535bf9dd-de3e-431d-8fc0-aae6cc5c66e9 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:08.012812 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-545582f8-4041-4fa8-b895-dfd96f2b4647 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16816b3849604454b4b277158f5cfa65'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:08.019238 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e08eb-dee3-4851-9e02-54d08f951afd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932d3076fb4b426d8c83d02eb60d8063', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:08.020424 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e08eb-dee3-4851-9e02-54d08f951afd service placement] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:08.021042 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e08eb-dee3-4851-9e02-54d08f951afd service placement] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:08.021752 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e08eb-dee3-4851-9e02-54d08f951afd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:08.025024 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-545582f8-4041-4fa8-b895-dfd96f2b4647 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16816b3849604454b4b277158f5cfa65'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:08.025620 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-545582f8-4041-4fa8-b895-dfd96f2b4647 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16816b3849604454b4b277158f5cfa65'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:08.037769 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e08eb-dee3-4851-9e02-54d08f951afd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:08.040604 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e08eb-dee3-4851-9e02-54d08f951afd service placement] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:08.041003 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-545582f8-4041-4fa8-b895-dfd96f2b4647 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16816b3849604454b4b277158f5cfa65'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:08.058586 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-545582f8-4041-4fa8-b895-dfd96f2b4647 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16816b3849604454b4b277158f5cfa65'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:08.070114 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 704/1403] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:07 2022] DELETE /identity/v3/users/16816b3849604454b4b277158f5cfa65 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:08.080308 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 700/1404] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:08.545378 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77385edd-7f96-4ea6-a73f-f3da2468b639 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:08.545875 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77385edd-7f96-4ea6-a73f-f3da2468b639 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:08.545973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77385edd-7f96-4ea6-a73f-f3da2468b639 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:08.548310 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 705/1405] 10.222.0.30 () {54 vars in 912 bytes} [Wed Nov 2 07:28:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:28:08.568849 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf6445-d76b-4499-a47a-2eae37663d1c None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:08.570797 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf6445-d76b-4499-a47a-2eae37663d1c None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:08.571372 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf6445-d76b-4499-a47a-2eae37663d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:08.656283 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-eabf6445-d76b-4499-a47a-2eae37663d1c None None] MFA Rules not processed for user `65d51ec74e1e4c25bcc4365eb858bac0`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:08.658734 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-eabf6445-d76b-4499-a47a-2eae37663d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:08.665670 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77385edd-7f96-4ea6-a73f-f3da2468b639 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:08.668713 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-77385edd-7f96-4ea6-a73f-f3da2468b639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:08.803362 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:08.813343 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 701/1406] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Nov 2 07:28:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:08.814958 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf6445-d76b-4499-a47a-2eae37663d1c None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:08.816392 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-eabf6445-d76b-4499-a47a-2eae37663d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:08.834979 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:08.837979 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77385edd-7f96-4ea6-a73f-f3da2468b639 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:08.843660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58494dd6-48fc-4bd4-aad2-f7f16446c21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:08.844722 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58494dd6-48fc-4bd4-aad2-f7f16446c21b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:08.845359 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58494dd6-48fc-4bd4-aad2-f7f16446c21b admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:08.846101 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58494dd6-48fc-4bd4-aad2-f7f16446c21b admin admin] PATH_INFO: `/v3/projects/b7c9e9f986054f9fbe25e4fe6f4b700a` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:08.847952 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58494dd6-48fc-4bd4-aad2-f7f16446c21b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7c9e9f986054f9fbe25e4fe6f4b700a)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:08.850438 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58494dd6-48fc-4bd4-aad2-f7f16446c21b admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:08.933614 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-58494dd6-48fc-4bd4-aad2-f7f16446c21b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7c9e9f986054f9fbe25e4fe6f4b700a', 'request_id': 'req-58494dd6-48fc-4bd4-aad2-f7f16446c21b'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:08.949293 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-58494dd6-48fc-4bd4-aad2-f7f16446c21b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7c9e9f986054f9fbe25e4fe6f4b700a', 'request_id': 'req-58494dd6-48fc-4bd4-aad2-f7f16446c21b'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:08.959717 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-58494dd6-48fc-4bd4-aad2-f7f16446c21b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7c9e9f986054f9fbe25e4fe6f4b700a', 'request_id': 'req-58494dd6-48fc-4bd4-aad2-f7f16446c21b'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:08.981213 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-58494dd6-48fc-4bd4-aad2-f7f16446c21b admin admin] The token cache is being invalidate because project b7c9e9f986054f9fbe25e4fe6f4b700a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:08.981648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-58494dd6-48fc-4bd4-aad2-f7f16446c21b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:08.982335 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-58494dd6-48fc-4bd4-aad2-f7f16446c21b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:08.988412 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 706/1407] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:08 2022] DELETE /identity/v3/projects/b7c9e9f986054f9fbe25e4fe6f4b700a => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:08.990523 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58494dd6-48fc-4bd4-aad2-f7f16446c21b admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:08.991391 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-58494dd6-48fc-4bd4-aad2-f7f16446c21b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:08.995343 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf6445-d76b-4499-a47a-2eae37663d1c None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:09.083641 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58494dd6-48fc-4bd4-aad2-f7f16446c21b admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:09.088227 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79c911-c122-4321-bd85-569202ba49c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:09.089077 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79c911-c122-4321-bd85-569202ba49c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:09.090291 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79c911-c122-4321-bd85-569202ba49c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:09.090291 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79c911-c122-4321-bd85-569202ba49c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:09.092161 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-6c79c911-c122-4321-bd85-569202ba49c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:09.108258 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c79c911-c122-4321-bd85-569202ba49c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:09.109746 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c79c911-c122-4321-bd85-569202ba49c0 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:09.120349 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 707/1408] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:09.121687 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ee16a-7acd-4628-9634-75fbed366665 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:09.121963 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ee16a-7acd-4628-9634-75fbed366665 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:09.122734 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ee16a-7acd-4628-9634-75fbed366665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:09.147027 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-ab9ee16a-7acd-4628-9634-75fbed366665 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:09.148496 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-ab9ee16a-7acd-4628-9634-75fbed366665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:09.151712 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e278a-ed5b-439e-89b5-f207551755ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:09.152373 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e278a-ed5b-439e-89b5-f207551755ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:09.152647 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e278a-ed5b-439e-89b5-f207551755ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:09.152915 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e278a-ed5b-439e-89b5-f207551755ef admin admin] PATH_INFO: `/v3/projects/8e238188c8004e3fb7d2cc03d0da9c85` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:09.154936 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e278a-ed5b-439e-89b5-f207551755ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e238188c8004e3fb7d2cc03d0da9c85)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:09.156863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e278a-ed5b-439e-89b5-f207551755ef admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:09.165055 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 708/1409] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:09.166620 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559963f3-2ad8-431c-b94d-8aa01c5ae5f6 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:09.167006 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559963f3-2ad8-431c-b94d-8aa01c5ae5f6 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:09.167369 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559963f3-2ad8-431c-b94d-8aa01c5ae5f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:09.192936 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-559963f3-2ad8-431c-b94d-8aa01c5ae5f6 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:09.194505 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-559963f3-2ad8-431c-b94d-8aa01c5ae5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:09.212649 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 709/1410] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:09.213654 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559963f3-2ad8-431c-b94d-8aa01c5ae5f6 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:09.245607 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559963f3-2ad8-431c-b94d-8aa01c5ae5f6 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:09.252640 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2be72f-f7a0-4e12-860b-8f8fa44f41d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:09.253522 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2be72f-f7a0-4e12-860b-8f8fa44f41d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:09.254835 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2be72f-f7a0-4e12-860b-8f8fa44f41d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:09.255512 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2be72f-f7a0-4e12-860b-8f8fa44f41d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:09.256512 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2be72f-f7a0-4e12-860b-8f8fa44f41d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:09.256942 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2be72f-f7a0-4e12-860b-8f8fa44f41d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:09.259306 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2be72f-f7a0-4e12-860b-8f8fa44f41d0 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:09.278481 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-072e278a-ed5b-439e-89b5-f207551755ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e238188c8004e3fb7d2cc03d0da9c85', 'request_id': 'req-072e278a-ed5b-439e-89b5-f207551755ef'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:09.300226 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-072e278a-ed5b-439e-89b5-f207551755ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e238188c8004e3fb7d2cc03d0da9c85', 'request_id': 'req-072e278a-ed5b-439e-89b5-f207551755ef'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:09.309616 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 710/1411] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:28:09.310919 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2be72f-f7a0-4e12-860b-8f8fa44f41d0 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:09.313171 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-7a2be72f-f7a0-4e12-860b-8f8fa44f41d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:09.313444 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-072e278a-ed5b-439e-89b5-f207551755ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e238188c8004e3fb7d2cc03d0da9c85', 'request_id': 'req-072e278a-ed5b-439e-89b5-f207551755ef'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:09.343928 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-072e278a-ed5b-439e-89b5-f207551755ef admin admin] The token cache is being invalidate because project 8e238188c8004e3fb7d2cc03d0da9c85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:09.344261 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-072e278a-ed5b-439e-89b5-f207551755ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:09.345486 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2be72f-f7a0-4e12-860b-8f8fa44f41d0 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:09.345892 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-072e278a-ed5b-439e-89b5-f207551755ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:09.352851 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 702/1412] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:08 2022] DELETE /identity/v3/projects/8e238188c8004e3fb7d2cc03d0da9c85 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:09.355153 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e278a-ed5b-439e-89b5-f207551755ef admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:09.357856 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-072e278a-ed5b-439e-89b5-f207551755ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:09.424673 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:09.441474 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:09.454981 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e278a-ed5b-439e-89b5-f207551755ef admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:09.462935 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f3a52-2428-4e97-8f3b-5dc2be8fd299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:09.463821 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f3a52-2428-4e97-8f3b-5dc2be8fd299 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:09.464311 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f3a52-2428-4e97-8f3b-5dc2be8fd299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:09.465611 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f3a52-2428-4e97-8f3b-5dc2be8fd299 admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:09.466627 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997f3a52-2428-4e97-8f3b-5dc2be8fd299 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:09.468817 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997f3a52-2428-4e97-8f3b-5dc2be8fd299 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:09.498928 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7795f-6671-4308-acf7-223e7b13861d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:09.501467 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7795f-6671-4308-acf7-223e7b13861d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:09.503517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7795f-6671-4308-acf7-223e7b13861d admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:09.504585 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7795f-6671-4308-acf7-223e7b13861d admin admin] PATH_INFO: `/v3/projects/9a9f191b424a43e4bc897e33be76e68e` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:09.508267 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a7795f-6671-4308-acf7-223e7b13861d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a9f191b424a43e4bc897e33be76e68e)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:09.513769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a7795f-6671-4308-acf7-223e7b13861d admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:09.528396 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 703/1413] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:09 2022] POST /identity/v3/projects => generated 373 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:09.528396 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f3a52-2428-4e97-8f3b-5dc2be8fd299 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:09.534589 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-997f3a52-2428-4e97-8f3b-5dc2be8fd299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:09.627718 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c3a7795f-6671-4308-acf7-223e7b13861d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a9f191b424a43e4bc897e33be76e68e', 'request_id': 'req-c3a7795f-6671-4308-acf7-223e7b13861d'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:09.643907 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f3a52-2428-4e97-8f3b-5dc2be8fd299 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:09.647917 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c3a7795f-6671-4308-acf7-223e7b13861d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a9f191b424a43e4bc897e33be76e68e', 'request_id': 'req-c3a7795f-6671-4308-acf7-223e7b13861d'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:09.662827 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c3a7795f-6671-4308-acf7-223e7b13861d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a9f191b424a43e4bc897e33be76e68e', 'request_id': 'req-c3a7795f-6671-4308-acf7-223e7b13861d'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:09.678092 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c3a7795f-6671-4308-acf7-223e7b13861d admin admin] The token cache is being invalidate because project 9a9f191b424a43e4bc897e33be76e68e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:09.679323 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c3a7795f-6671-4308-acf7-223e7b13861d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:09.680272 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c3a7795f-6671-4308-acf7-223e7b13861d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:09.688778 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 711/1414] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:09 2022] DELETE /identity/v3/projects/9a9f191b424a43e4bc897e33be76e68e => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:09.688778 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7795f-6671-4308-acf7-223e7b13861d admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:09.688778 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-c3a7795f-6671-4308-acf7-223e7b13861d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:09.729403 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2093558b-0141-4063-aa2f-55f4c0bca476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:09.729973 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093558b-0141-4063-aa2f-55f4c0bca476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:09.730246 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093558b-0141-4063-aa2f-55f4c0bca476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:09.730522 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093558b-0141-4063-aa2f-55f4c0bca476 admin admin] PATH_INFO: `/v3/users/56c8bcb0e84149f89af893a698a3d9e1` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:09.732671 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2093558b-0141-4063-aa2f-55f4c0bca476 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56c8bcb0e84149f89af893a698a3d9e1)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:09.734946 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2093558b-0141-4063-aa2f-55f4c0bca476 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:09.845796 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-2093558b-0141-4063-aa2f-55f4c0bca476 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56c8bcb0e84149f89af893a698a3d9e1'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:09.846752 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-2093558b-0141-4063-aa2f-55f4c0bca476 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56c8bcb0e84149f89af893a698a3d9e1'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:09.856295 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-2093558b-0141-4063-aa2f-55f4c0bca476 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56c8bcb0e84149f89af893a698a3d9e1'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:09.856447 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7795f-6671-4308-acf7-223e7b13861d admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:09.868639 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c5297-4df9-40bf-8598-f660f350b461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:09.870842 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c5297-4df9-40bf-8598-f660f350b461 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:09.871629 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c5297-4df9-40bf-8598-f660f350b461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:09.872350 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-2093558b-0141-4063-aa2f-55f4c0bca476 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56c8bcb0e84149f89af893a698a3d9e1'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:09.872929 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c5297-4df9-40bf-8598-f660f350b461 admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:09.873770 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c5297-4df9-40bf-8598-f660f350b461 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:09.876179 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c5297-4df9-40bf-8598-f660f350b461 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:09.877479 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:09.885772 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-2093558b-0141-4063-aa2f-55f4c0bca476 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56c8bcb0e84149f89af893a698a3d9e1'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:09.911423 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 704/1415] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:09 2022] DELETE /identity/v3/users/56c8bcb0e84149f89af893a698a3d9e1 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:09.912480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2093558b-0141-4063-aa2f-55f4c0bca476 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:09.914077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-2093558b-0141-4063-aa2f-55f4c0bca476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:09.939316 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 712/1416] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:09 2022] POST /identity/v3/users => generated 287 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:09.941565 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c5297-4df9-40bf-8598-f660f350b461 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:09.948657 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-1a0c5297-4df9-40bf-8598-f660f350b461 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:09.995166 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c5297-4df9-40bf-8598-f660f350b461 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:10.056751 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2093558b-0141-4063-aa2f-55f4c0bca476 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:10.064304 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c89ac3-fcde-4de7-be00-648f3f5ca382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:10.065435 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c89ac3-fcde-4de7-be00-648f3f5ca382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:10.065850 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c89ac3-fcde-4de7-be00-648f3f5ca382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:10.066411 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c89ac3-fcde-4de7-be00-648f3f5ca382 admin admin] PATH_INFO: `/v3/users/2180d87d41784af8a9473ba59005ca06` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:10.068423 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c89ac3-fcde-4de7-be00-648f3f5ca382 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2180d87d41784af8a9473ba59005ca06)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:10.072536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c89ac3-fcde-4de7-be00-648f3f5ca382 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:10.194439 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8209106-9358-4ee8-9ff3-8feb1850698a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:10.196292 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8209106-9358-4ee8-9ff3-8feb1850698a admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:10.196588 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8209106-9358-4ee8-9ff3-8feb1850698a admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:10.199130 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8209106-9358-4ee8-9ff3-8feb1850698a admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:10.200052 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8209106-9358-4ee8-9ff3-8feb1850698a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:10.207642 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8209106-9358-4ee8-9ff3-8feb1850698a admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:10.221988 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-33c89ac3-fcde-4de7-be00-648f3f5ca382 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2180d87d41784af8a9473ba59005ca06'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:10.224209 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-33c89ac3-fcde-4de7-be00-648f3f5ca382 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2180d87d41784af8a9473ba59005ca06'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:10.242935 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-33c89ac3-fcde-4de7-be00-648f3f5ca382 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2180d87d41784af8a9473ba59005ca06'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:10.247332 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:10.269265 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 713/1417] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:10.269265 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8209106-9358-4ee8-9ff3-8feb1850698a admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:10.301001 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-c8209106-9358-4ee8-9ff3-8feb1850698a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:10.310532 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-33c89ac3-fcde-4de7-be00-648f3f5ca382 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2180d87d41784af8a9473ba59005ca06'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:10.327842 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-33c89ac3-fcde-4de7-be00-648f3f5ca382 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2180d87d41784af8a9473ba59005ca06'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:10.337990 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 705/1418] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:09 2022] DELETE /identity/v3/users/2180d87d41784af8a9473ba59005ca06 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:10.339997 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c89ac3-fcde-4de7-be00-648f3f5ca382 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:10.344584 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-33c89ac3-fcde-4de7-be00-648f3f5ca382 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:10.390353 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8209106-9358-4ee8-9ff3-8feb1850698a admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:10.449210 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da71a73-b358-492a-93e3-6015c1c7c946 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:10.449839 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da71a73-b358-492a-93e3-6015c1c7c946 service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:10.450110 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da71a73-b358-492a-93e3-6015c1c7c946 service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:10.450371 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da71a73-b358-492a-93e3-6015c1c7c946 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:10.452193 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-8da71a73-b358-492a-93e3-6015c1c7c946 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:10.499974 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c89ac3-fcde-4de7-be00-648f3f5ca382 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:10.532111 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c62f72-57f3-4be8-b495-fd0c8921530e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:10.533802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c62f72-57f3-4be8-b495-fd0c8921530e admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:10.533802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c62f72-57f3-4be8-b495-fd0c8921530e admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:10.533802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c62f72-57f3-4be8-b495-fd0c8921530e admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:10.534073 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c62f72-57f3-4be8-b495-fd0c8921530e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:10.543094 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c62f72-57f3-4be8-b495-fd0c8921530e admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:10.578785 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 706/1419] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:10.580950 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff5516-87ff-4c2d-9c25-0758bdf2fdcd None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:10.581664 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff5516-87ff-4c2d-9c25-0758bdf2fdcd None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:10.581664 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff5516-87ff-4c2d-9c25-0758bdf2fdcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:10.623226 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:10.633124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-7bff5516-87ff-4c2d-9c25-0758bdf2fdcd None None] MFA Rules not processed for user `d568635a644d444096997355ceb21202`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:10.635015 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-7bff5516-87ff-4c2d-9c25-0758bdf2fdcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:10.764235 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 707/1420] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:10.764235 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bff5516-87ff-4c2d-9c25-0758bdf2fdcd None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:10.770420 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-7bff5516-87ff-4c2d-9c25-0758bdf2fdcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:10.780117 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da71a73-b358-492a-93e3-6015c1c7c946 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:10.782676 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da71a73-b358-492a-93e3-6015c1c7c946 service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:10.815729 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 714/1421] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:10 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:10.820777 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f37a75-bacd-4c20-ac79-1c7c8c3e07e3 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:10.822191 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f37a75-bacd-4c20-ac79-1c7c8c3e07e3 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:10.822608 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f37a75-bacd-4c20-ac79-1c7c8c3e07e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:10.879251 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-03f37a75-bacd-4c20-ac79-1c7c8c3e07e3 None None] MFA Rules not processed for user `be586640013949589d2a7b11a7a2f3b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:10.880981 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-03f37a75-bacd-4c20-ac79-1c7c8c3e07e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:10.910202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bff5516-87ff-4c2d-9c25-0758bdf2fdcd None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:10.926778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e4f18-727f-4c61-9a7d-248d86e140e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932d3076fb4b426d8c83d02eb60d8063', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:10.927525 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e4f18-727f-4c61-9a7d-248d86e140e2 service placement] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:10.927813 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e4f18-727f-4c61-9a7d-248d86e140e2 service placement] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:10.928122 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e4f18-727f-4c61-9a7d-248d86e140e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:10.930830 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-9a4e4f18-727f-4c61-9a7d-248d86e140e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:10.966416 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:10.987770 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e4f18-727f-4c61-9a7d-248d86e140e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:10.987770 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e4f18-727f-4c61-9a7d-248d86e140e2 service placement] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:10.991022 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 715/1422] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:10.993137 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d2121-bd57-4782-9a7a-e390affa0fc9 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:10.993827 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d2121-bd57-4782-9a7a-e390affa0fc9 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:10.994323 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d2121-bd57-4782-9a7a-e390affa0fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:11.014589 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 708/1423] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:11.015401 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e4f18-727f-4c61-9a7d-248d86e140e2 service placement] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:11.034131 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e4f18-727f-4c61-9a7d-248d86e140e2 service placement] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:11.042338 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f489ebe-1787-4cf6-8b2b-bed044663892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:11.043200 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f489ebe-1787-4cf6-8b2b-bed044663892 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:11.044312 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f489ebe-1787-4cf6-8b2b-bed044663892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:11.044638 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f489ebe-1787-4cf6-8b2b-bed044663892 admin admin] PATH_INFO: `/v3/projects/71bf346a33bc4828bf5750e94ee1cbeb/users/dbcbff89b82d41a1b0188eff9f569144/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:11.053953 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-dd7d2121-bd57-4782-9a7a-e390affa0fc9 None None] MFA Rules not processed for user `e4f9bb9b60a246f3a84a4d4618c3f527`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:11.058613 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-dd7d2121-bd57-4782-9a7a-e390affa0fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:11.067160 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f489ebe-1787-4cf6-8b2b-bed044663892 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71bf346a33bc4828bf5750e94ee1cbeb, user_id=dbcbff89b82d41a1b0188eff9f569144, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:11.070852 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f489ebe-1787-4cf6-8b2b-bed044663892 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:11.117126 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 709/1424] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:11 2022] PUT /identity/v3/projects/71bf346a33bc4828bf5750e94ee1cbeb/users/dbcbff89b82d41a1b0188eff9f569144/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:11.123262 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f82b62-82b1-4f34-a725-77a2eefafbf3 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:11.124476 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f82b62-82b1-4f34-a725-77a2eefafbf3 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:11.125353 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f82b62-82b1-4f34-a725-77a2eefafbf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:11.155238 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 716/1425] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:10 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:11.160807 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d2121-bd57-4782-9a7a-e390affa0fc9 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:11.162862 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-dd7d2121-bd57-4782-9a7a-e390affa0fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:11.179126 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d2121-bd57-4782-9a7a-e390affa0fc9 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:11.197524 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-53f82b62-82b1-4f34-a725-77a2eefafbf3 None None] MFA Rules not processed for user `d568635a644d444096997355ceb21202`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:11.201049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-53f82b62-82b1-4f34-a725-77a2eefafbf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:11.269768 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41684348-d813-4827-8650-7ce0e5f116c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:11.270712 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41684348-d813-4827-8650-7ce0e5f116c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:11.271119 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41684348-d813-4827-8650-7ce0e5f116c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:11.271534 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41684348-d813-4827-8650-7ce0e5f116c0 admin admin] PATH_INFO: `/v3/projects/6d9119f458fb430482e054cca8b1fcd4` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:11.273516 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41684348-d813-4827-8650-7ce0e5f116c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d9119f458fb430482e054cca8b1fcd4)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:11.280475 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41684348-d813-4827-8650-7ce0e5f116c0 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:11.402993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 710/1426] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:11.403922 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f82b62-82b1-4f34-a725-77a2eefafbf3 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:11.406379 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-53f82b62-82b1-4f34-a725-77a2eefafbf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:11.408500 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-41684348-d813-4827-8650-7ce0e5f116c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d9119f458fb430482e054cca8b1fcd4', 'request_id': 'req-41684348-d813-4827-8650-7ce0e5f116c0'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:11.434039 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-41684348-d813-4827-8650-7ce0e5f116c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d9119f458fb430482e054cca8b1fcd4', 'request_id': 'req-41684348-d813-4827-8650-7ce0e5f116c0'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:11.456358 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-41684348-d813-4827-8650-7ce0e5f116c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d9119f458fb430482e054cca8b1fcd4', 'request_id': 'req-41684348-d813-4827-8650-7ce0e5f116c0'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:11.471327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f82b62-82b1-4f34-a725-77a2eefafbf3 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:11.507026 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:11.510183 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-41684348-d813-4827-8650-7ce0e5f116c0 admin admin] The token cache is being invalidate because project 6d9119f458fb430482e054cca8b1fcd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:11.510484 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-41684348-d813-4827-8650-7ce0e5f116c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:11.511539 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-41684348-d813-4827-8650-7ce0e5f116c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:11.518760 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 717/1427] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:11 2022] DELETE /identity/v3/projects/6d9119f458fb430482e054cca8b1fcd4 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:11.518760 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41684348-d813-4827-8650-7ce0e5f116c0 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:11.520324 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-41684348-d813-4827-8650-7ce0e5f116c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:11.612593 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:11.674621 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41684348-d813-4827-8650-7ce0e5f116c0 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:11.685142 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf25cd39-efba-414b-a193-a79b9dfbaa4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:11.685847 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25cd39-efba-414b-a193-a79b9dfbaa4b service nova] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:11.686126 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25cd39-efba-414b-a193-a79b9dfbaa4b service nova] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:11.686397 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25cd39-efba-414b-a193-a79b9dfbaa4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:11.689723 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-cf25cd39-efba-414b-a193-a79b9dfbaa4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:11.741444 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd8193-8680-458a-ae6a-ebac5b765a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:11.743026 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd8193-8680-458a-ae6a-ebac5b765a33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:11.743776 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd8193-8680-458a-ae6a-ebac5b765a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:11.744515 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd8193-8680-458a-ae6a-ebac5b765a33 admin admin] PATH_INFO: `/v3/projects/3a667c5fa8ee4c73ae7130e24a55ce16` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:11.748665 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd8193-8680-458a-ae6a-ebac5b765a33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a667c5fa8ee4c73ae7130e24a55ce16)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:11.755268 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd8193-8680-458a-ae6a-ebac5b765a33 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:11.784553 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf25cd39-efba-414b-a193-a79b9dfbaa4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:11.787051 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf25cd39-efba-414b-a193-a79b9dfbaa4b service nova] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:11.816707 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 718/1428] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:11.822348 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee607fc-1c5e-46b0-aef0-eada2a2aa026 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:11.822828 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee607fc-1c5e-46b0-aef0-eada2a2aa026 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:11.824141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee607fc-1c5e-46b0-aef0-eada2a2aa026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:11.875587 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-3ee607fc-1c5e-46b0-aef0-eada2a2aa026 None None] MFA Rules not processed for user `dbcbff89b82d41a1b0188eff9f569144`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:11.878463 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-3ee607fc-1c5e-46b0-aef0-eada2a2aa026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:12.053292 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 719/1429] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:12.055641 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f12764-e086-4bf6-b40f-6cd4639f5047 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:12.056114 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f12764-e086-4bf6-b40f-6cd4639f5047 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:12.056583 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f12764-e086-4bf6-b40f-6cd4639f5047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:12.131770 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-76f12764-e086-4bf6-b40f-6cd4639f5047 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:12.134483 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-76f12764-e086-4bf6-b40f-6cd4639f5047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:12.178712 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 720/1430] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:12.178712 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f12764-e086-4bf6-b40f-6cd4639f5047 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:12.178712 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-76f12764-e086-4bf6-b40f-6cd4639f5047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:12.219138 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f12764-e086-4bf6-b40f-6cd4639f5047 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:12.239058 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f2e1c-40b7-46e6-87c3-dcdd1e0e45b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:12.239723 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f2e1c-40b7-46e6-87c3-dcdd1e0e45b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:12.240009 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f2e1c-40b7-46e6-87c3-dcdd1e0e45b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:12.241673 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f2e1c-40b7-46e6-87c3-dcdd1e0e45b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:12.243425 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f2e1c-40b7-46e6-87c3-dcdd1e0e45b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:12.245467 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f2e1c-40b7-46e6-87c3-dcdd1e0e45b6 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:12.248464 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-aecd8193-8680-458a-ae6a-ebac5b765a33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a667c5fa8ee4c73ae7130e24a55ce16', 'request_id': 'req-aecd8193-8680-458a-ae6a-ebac5b765a33'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:12.265610 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-aecd8193-8680-458a-ae6a-ebac5b765a33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a667c5fa8ee4c73ae7130e24a55ce16', 'request_id': 'req-aecd8193-8680-458a-ae6a-ebac5b765a33'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:12.272224 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-aecd8193-8680-458a-ae6a-ebac5b765a33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a667c5fa8ee4c73ae7130e24a55ce16', 'request_id': 'req-aecd8193-8680-458a-ae6a-ebac5b765a33'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:12.294205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-aecd8193-8680-458a-ae6a-ebac5b765a33 admin admin] The token cache is being invalidate because project 3a667c5fa8ee4c73ae7130e24a55ce16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:12.294434 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-aecd8193-8680-458a-ae6a-ebac5b765a33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:12.295133 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-aecd8193-8680-458a-ae6a-ebac5b765a33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:12.301598 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 711/1431] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:11 2022] DELETE /identity/v3/projects/3a667c5fa8ee4c73ae7130e24a55ce16 => generated 0 bytes in 893 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:12.301598 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd8193-8680-458a-ae6a-ebac5b765a33 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:12.301598 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-aecd8193-8680-458a-ae6a-ebac5b765a33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:12.337944 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 721/1432] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:12 2022] POST /identity/v3/projects => generated 371 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:12.341104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac16a5-0521-4f45-8d53-8c95f42db70d None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:12.341385 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac16a5-0521-4f45-8d53-8c95f42db70d None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:12.343507 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac16a5-0521-4f45-8d53-8c95f42db70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:12.389370 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-23ac16a5-0521-4f45-8d53-8c95f42db70d None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:12.391976 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-23ac16a5-0521-4f45-8d53-8c95f42db70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:12.450103 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:12.504807 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd8193-8680-458a-ae6a-ebac5b765a33 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:12.547841 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 722/1433] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:12.550587 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba229920-0edf-452b-9d55-c46609fa6fa4 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:12.551044 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba229920-0edf-452b-9d55-c46609fa6fa4 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:12.552797 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba229920-0edf-452b-9d55-c46609fa6fa4 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:12.555162 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 723/1434] 10.222.0.30 () {54 vars in 912 bytes} [Wed Nov 2 07:28:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:28:12.556540 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddde8e5-c3ad-4c3d-a8f6-5ef721bc6838 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:12.557091 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddde8e5-c3ad-4c3d-a8f6-5ef721bc6838 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:12.557743 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddde8e5-c3ad-4c3d-a8f6-5ef721bc6838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:12.584138 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-4ddde8e5-c3ad-4c3d-a8f6-5ef721bc6838 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:12.586035 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-4ddde8e5-c3ad-4c3d-a8f6-5ef721bc6838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:12.610852 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:12.615083 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 724/1435] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:12.617712 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddde8e5-c3ad-4c3d-a8f6-5ef721bc6838 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:12.623558 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-4ddde8e5-c3ad-4c3d-a8f6-5ef721bc6838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:12.654741 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddde8e5-c3ad-4c3d-a8f6-5ef721bc6838 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:12.663542 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba268e0-0ba0-4a63-9310-b35e7c393dd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:12.664784 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba268e0-0ba0-4a63-9310-b35e7c393dd3 service nova] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:12.665406 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba268e0-0ba0-4a63-9310-b35e7c393dd3 service nova] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:12.667046 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e915d-72c2-4d75-8f97-efa542e46b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:12.667651 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba268e0-0ba0-4a63-9310-b35e7c393dd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:12.668912 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e915d-72c2-4d75-8f97-efa542e46b64 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:12.669634 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e915d-72c2-4d75-8f97-efa542e46b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:12.671684 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e915d-72c2-4d75-8f97-efa542e46b64 admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:12.672311 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-bba268e0-0ba0-4a63-9310-b35e7c393dd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:12.673368 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e915d-72c2-4d75-8f97-efa542e46b64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:12.677040 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e915d-72c2-4d75-8f97-efa542e46b64 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:12.733604 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 725/1436] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:12 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:12.735910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e915d-72c2-4d75-8f97-efa542e46b64 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:12.760066 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e915d-72c2-4d75-8f97-efa542e46b64 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:12.778053 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fba73f-b2c0-46e5-9817-00ae5375c47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:12.779937 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fba73f-b2c0-46e5-9817-00ae5375c47f admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:12.781401 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fba73f-b2c0-46e5-9817-00ae5375c47f admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:12.781771 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fba73f-b2c0-46e5-9817-00ae5375c47f admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:12.784138 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fba73f-b2c0-46e5-9817-00ae5375c47f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:12.787042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fba73f-b2c0-46e5-9817-00ae5375c47f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:12.794056 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fba73f-b2c0-46e5-9817-00ae5375c47f admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:12.843649 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 726/1437] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:28:12.847360 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf648d-e085-4bfa-9110-ae236a3c1927 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:12.852402 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf648d-e085-4bfa-9110-ae236a3c1927 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:12.853416 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf648d-e085-4bfa-9110-ae236a3c1927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:12.879062 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba268e0-0ba0-4a63-9310-b35e7c393dd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:12.882531 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba268e0-0ba0-4a63-9310-b35e7c393dd3 service nova] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:12.924494 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 712/1438] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:12 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:12.928810 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2474bd-0b07-44bb-9f3e-b99180fa8926 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:12.929100 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2474bd-0b07-44bb-9f3e-b99180fa8926 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:12.929369 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2474bd-0b07-44bb-9f3e-b99180fa8926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:12.936047 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-1ebf648d-e085-4bfa-9110-ae236a3c1927 None None] MFA Rules not processed for user `65d51ec74e1e4c25bcc4365eb858bac0`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:12.942623 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-1ebf648d-e085-4bfa-9110-ae236a3c1927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:13.019951 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-6c2474bd-0b07-44bb-9f3e-b99180fa8926 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:13.024257 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-6c2474bd-0b07-44bb-9f3e-b99180fa8926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:13.054878 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 713/1439] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:13.060091 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2474bd-0b07-44bb-9f3e-b99180fa8926 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:13.098046 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 727/1440] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Nov 2 07:28:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:13.099458 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf648d-e085-4bfa-9110-ae236a3c1927 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:13.105549 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2474bd-0b07-44bb-9f3e-b99180fa8926 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:13.116891 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf648d-e085-4bfa-9110-ae236a3c1927 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:13.123262 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289eecde-afdc-4e63-88f7-81d475d5786e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:13.123761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aad30f-064f-427d-ac39-e69a4ac916f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:13.123889 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289eecde-afdc-4e63-88f7-81d475d5786e admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:13.124158 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289eecde-afdc-4e63-88f7-81d475d5786e admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:13.124426 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289eecde-afdc-4e63-88f7-81d475d5786e admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:13.125097 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aad30f-064f-427d-ac39-e69a4ac916f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:13.125562 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aad30f-064f-427d-ac39-e69a4ac916f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:13.126041 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aad30f-064f-427d-ac39-e69a4ac916f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:13.126903 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aad30f-064f-427d-ac39-e69a4ac916f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:13.129699 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aad30f-064f-427d-ac39-e69a4ac916f9 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:13.133016 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289eecde-afdc-4e63-88f7-81d475d5786e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:13.138370 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289eecde-afdc-4e63-88f7-81d475d5786e admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:13.170378 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 714/1441] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:13 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:13.171168 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289eecde-afdc-4e63-88f7-81d475d5786e admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:13.183800 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 728/1442] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:13 2022] POST /identity/v3/projects => generated 367 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:13.188419 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aad30f-064f-427d-ac39-e69a4ac916f9 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:13.190399 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-b7aad30f-064f-427d-ac39-e69a4ac916f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:13.202598 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289eecde-afdc-4e63-88f7-81d475d5786e admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:13.279478 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aad30f-064f-427d-ac39-e69a4ac916f9 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:13.284480 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a40b3-a632-4743-b95d-a1188d8d27b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:13.284480 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a40b3-a632-4743-b95d-a1188d8d27b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:13.284792 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a40b3-a632-4743-b95d-a1188d8d27b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:13.284865 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a40b3-a632-4743-b95d-a1188d8d27b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:13.297883 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a40b3-a632-4743-b95d-a1188d8d27b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:13.297883 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a40b3-a632-4743-b95d-a1188d8d27b3 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:13.307876 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 729/1443] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:13.308901 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a40b3-a632-4743-b95d-a1188d8d27b3 service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:13.331953 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961793f-574e-4dff-ab07-1817f765e3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:13.332674 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961793f-574e-4dff-ab07-1817f765e3e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:13.332953 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961793f-574e-4dff-ab07-1817f765e3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:13.333297 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961793f-574e-4dff-ab07-1817f765e3e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:13.334000 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961793f-574e-4dff-ab07-1817f765e3e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:13.334306 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961793f-574e-4dff-ab07-1817f765e3e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:13.339291 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961793f-574e-4dff-ab07-1817f765e3e3 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:13.349567 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:13.377622 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 715/1444] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:28:13.378649 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961793f-574e-4dff-ab07-1817f765e3e3 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:13.389718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961793f-574e-4dff-ab07-1817f765e3e3 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:13.402235 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00132a5a-3bbc-4c0b-92c7-3f2d56d654e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:13.402908 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00132a5a-3bbc-4c0b-92c7-3f2d56d654e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:13.403183 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00132a5a-3bbc-4c0b-92c7-3f2d56d654e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:13.408941 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00132a5a-3bbc-4c0b-92c7-3f2d56d654e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:13.408941 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00132a5a-3bbc-4c0b-92c7-3f2d56d654e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:13.408941 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00132a5a-3bbc-4c0b-92c7-3f2d56d654e6 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:13.418275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a40b3-a632-4743-b95d-a1188d8d27b3 service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:13.431810 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1ed73-74ca-4a6a-96a7-3f7972989e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:13.432829 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1ed73-74ca-4a6a-96a7-3f7972989e90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:13.433167 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1ed73-74ca-4a6a-96a7-3f7972989e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:13.433512 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1ed73-74ca-4a6a-96a7-3f7972989e90 admin admin] PATH_INFO: `/v3/projects/7da9159cbb5448fb9361edd9e92f2713/users/774fe1201baa4a0b922a35dacb419f38/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:13.442885 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 716/1445] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:13 2022] POST /identity/v3/users => generated 284 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:13.444613 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00132a5a-3bbc-4c0b-92c7-3f2d56d654e6 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:13.468570 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00132a5a-3bbc-4c0b-92c7-3f2d56d654e6 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:13.480742 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a767108-f44d-468c-817a-a895ce55d460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:13.480980 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1ed73-74ca-4a6a-96a7-3f7972989e90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7da9159cbb5448fb9361edd9e92f2713, user_id=774fe1201baa4a0b922a35dacb419f38, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:13.482871 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a767108-f44d-468c-817a-a895ce55d460 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:13.483158 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a767108-f44d-468c-817a-a895ce55d460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:13.486868 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1ed73-74ca-4a6a-96a7-3f7972989e90 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:13.487039 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a767108-f44d-468c-817a-a895ce55d460 admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:13.487571 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a767108-f44d-468c-817a-a895ce55d460 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:13.490377 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a767108-f44d-468c-817a-a895ce55d460 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:13.506186 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 730/1446] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:13 2022] PUT /identity/v3/projects/7da9159cbb5448fb9361edd9e92f2713/users/774fe1201baa4a0b922a35dacb419f38/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:13.509866 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1ed73-74ca-4a6a-96a7-3f7972989e90 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:13.577291 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 717/1447] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:13 2022] POST /identity/v3/projects => generated 365 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:13.580037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a767108-f44d-468c-817a-a895ce55d460 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:13.622452 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:13.642631 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1ed73-74ca-4a6a-96a7-3f7972989e90 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:13.735441 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9799cc-6ffa-43b0-8ff4-1828c32469e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:13.736071 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9799cc-6ffa-43b0-8ff4-1828c32469e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:13.736345 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9799cc-6ffa-43b0-8ff4-1828c32469e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:13.736615 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9799cc-6ffa-43b0-8ff4-1828c32469e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:13.737271 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9799cc-6ffa-43b0-8ff4-1828c32469e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:13.738938 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9799cc-6ffa-43b0-8ff4-1828c32469e4 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:13.757200 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a767108-f44d-468c-817a-a895ce55d460 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:13.766505 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 731/1448] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:13 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:13.767984 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9799cc-6ffa-43b0-8ff4-1828c32469e4 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:13.779164 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b4af0-5eda-4411-8ebf-6fe378a63b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:13.779856 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b4af0-5eda-4411-8ebf-6fe378a63b77 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:13.780133 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b4af0-5eda-4411-8ebf-6fe378a63b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:13.780403 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b4af0-5eda-4411-8ebf-6fe378a63b77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:13.781076 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b4af0-5eda-4411-8ebf-6fe378a63b77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:13.781449 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9799cc-6ffa-43b0-8ff4-1828c32469e4 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:13.789090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b4af0-5eda-4411-8ebf-6fe378a63b77 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:13.795390 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b179cf5e-3fde-43b3-b5dc-6a42916b499d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:13.799085 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179cf5e-3fde-43b3-b5dc-6a42916b499d admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:13.799085 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179cf5e-3fde-43b3-b5dc-6a42916b499d admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:13.799085 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179cf5e-3fde-43b3-b5dc-6a42916b499d admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:13.799085 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b179cf5e-3fde-43b3-b5dc-6a42916b499d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:13.799469 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b179cf5e-3fde-43b3-b5dc-6a42916b499d admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:13.822455 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 718/1449] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:13 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:13.824026 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b4af0-5eda-4411-8ebf-6fe378a63b77 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:13.840934 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b4af0-5eda-4411-8ebf-6fe378a63b77 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:13.848225 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae98615-a90d-44e6-bfda-9adf4cf7888f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:13.848728 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae98615-a90d-44e6-bfda-9adf4cf7888f admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:13.849009 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae98615-a90d-44e6-bfda-9adf4cf7888f admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:13.849264 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae98615-a90d-44e6-bfda-9adf4cf7888f admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:13.849905 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae98615-a90d-44e6-bfda-9adf4cf7888f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:13.851567 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae98615-a90d-44e6-bfda-9adf4cf7888f admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:13.859751 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 732/1450] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:13 2022] POST /identity/v3/users => generated 283 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:13.860907 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b179cf5e-3fde-43b3-b5dc-6a42916b499d admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:13.867427 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 719/1451] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:13.867427 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae98615-a90d-44e6-bfda-9adf4cf7888f admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:13.878011 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae98615-a90d-44e6-bfda-9adf4cf7888f admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:13.890680 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b179cf5e-3fde-43b3-b5dc-6a42916b499d admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:13.893237 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10c31a-11df-46c9-880d-08927857796b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:13.894773 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10c31a-11df-46c9-880d-08927857796b admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:13.895059 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10c31a-11df-46c9-880d-08927857796b admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:13.895334 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10c31a-11df-46c9-880d-08927857796b admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:13.897329 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec10c31a-11df-46c9-880d-08927857796b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:13.902167 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec10c31a-11df-46c9-880d-08927857796b admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:13.904205 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0490424-f471-4a0b-997d-59a9da9e53a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:13.905116 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0490424-f471-4a0b-997d-59a9da9e53a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:13.905116 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0490424-f471-4a0b-997d-59a9da9e53a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:13.905236 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0490424-f471-4a0b-997d-59a9da9e53a5 admin admin] PATH_INFO: `/v3/projects/7da9159cbb5448fb9361edd9e92f2713/users/774fe1201baa4a0b922a35dacb419f38/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:13.911826 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0490424-f471-4a0b-997d-59a9da9e53a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7da9159cbb5448fb9361edd9e92f2713, user_id=774fe1201baa4a0b922a35dacb419f38, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:13.915379 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0490424-f471-4a0b-997d-59a9da9e53a5 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:13.932650 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 720/1452] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:13 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:13.933575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10c31a-11df-46c9-880d-08927857796b admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:13.945964 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 733/1453] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:13 2022] PUT /identity/v3/projects/7da9159cbb5448fb9361edd9e92f2713/users/774fe1201baa4a0b922a35dacb419f38/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:13.955619 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0490424-f471-4a0b-997d-59a9da9e53a5 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:13.979703 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10c31a-11df-46c9-880d-08927857796b admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:14.026591 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:14.167547 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7945df8-f8c1-4cff-a289-2d0f84a9af2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:14.173795 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7945df8-f8c1-4cff-a289-2d0f84a9af2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:14.174075 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7945df8-f8c1-4cff-a289-2d0f84a9af2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:14.174445 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7945df8-f8c1-4cff-a289-2d0f84a9af2d admin admin] PATH_INFO: `/v3/projects/ddfd598da25f41269dcd457b1120d08a/users/b1374a56b69c4ccfa0cc92ee3b53493a/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:14.196523 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0490424-f471-4a0b-997d-59a9da9e53a5 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:14.212629 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e82656-bacf-4778-ae5d-17ab56f201f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:14.213277 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e82656-bacf-4778-ae5d-17ab56f201f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:14.213556 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e82656-bacf-4778-ae5d-17ab56f201f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:14.213830 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e82656-bacf-4778-ae5d-17ab56f201f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:14.214636 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e82656-bacf-4778-ae5d-17ab56f201f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:14.216305 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e82656-bacf-4778-ae5d-17ab56f201f3 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:14.228756 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7945df8-f8c1-4cff-a289-2d0f84a9af2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddfd598da25f41269dcd457b1120d08a, user_id=b1374a56b69c4ccfa0cc92ee3b53493a, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:14.232573 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7945df8-f8c1-4cff-a289-2d0f84a9af2d admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:14.260837 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 721/1454] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:13 2022] PUT /identity/v3/projects/ddfd598da25f41269dcd457b1120d08a/users/b1374a56b69c4ccfa0cc92ee3b53493a/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:14.262819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf79ea-e37d-4106-940f-357000c50145 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:14.264828 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf79ea-e37d-4106-940f-357000c50145 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:14.265112 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf79ea-e37d-4106-940f-357000c50145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:14.267650 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 734/1455] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:13 2022] GET /identity/v3/roles => generated 1420 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:14.272581 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e82656-bacf-4778-ae5d-17ab56f201f3 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:14.313074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e82656-bacf-4778-ae5d-17ab56f201f3 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:14.324090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-ddaf79ea-e37d-4106-940f-357000c50145 None None] MFA Rules not processed for user `774fe1201baa4a0b922a35dacb419f38`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:14.325646 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-ddaf79ea-e37d-4106-940f-357000c50145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:14.459565 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31dd83-b7f7-4bbc-a166-63af58d17905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:14.460497 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31dd83-b7f7-4bbc-a166-63af58d17905 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:14.460794 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31dd83-b7f7-4bbc-a166-63af58d17905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:14.461413 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31dd83-b7f7-4bbc-a166-63af58d17905 admin admin] PATH_INFO: `/v3/projects/e72cb1596f56407cada971aaeb7fd599/users/7538b9215f2848408b9b9282dccebd34/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:14.499683 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e31dd83-b7f7-4bbc-a166-63af58d17905 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e72cb1596f56407cada971aaeb7fd599, user_id=7538b9215f2848408b9b9282dccebd34, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:14.504607 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e31dd83-b7f7-4bbc-a166-63af58d17905 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:14.534991 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:14.555568 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 735/1456] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:14 2022] PUT /identity/v3/projects/e72cb1596f56407cada971aaeb7fd599/users/7538b9215f2848408b9b9282dccebd34/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:14.559946 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 722/1457] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:14 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:14.561458 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442a859-1f86-4e09-a8e4-950f968c70d0 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:14.561734 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442a859-1f86-4e09-a8e4-950f968c70d0 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:14.561998 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442a859-1f86-4e09-a8e4-950f968c70d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:14.566494 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096357a2-a560-45b4-8558-8d2ed3491ba8 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:14.566834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096357a2-a560-45b4-8558-8d2ed3491ba8 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:14.567188 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096357a2-a560-45b4-8558-8d2ed3491ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:14.617282 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-096357a2-a560-45b4-8558-8d2ed3491ba8 None None] MFA Rules not processed for user `b1374a56b69c4ccfa0cc92ee3b53493a`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:14.618670 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-096357a2-a560-45b4-8558-8d2ed3491ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:14.670998 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-6442a859-1f86-4e09-a8e4-950f968c70d0 None None] MFA Rules not processed for user `7538b9215f2848408b9b9282dccebd34`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:14.672241 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-6442a859-1f86-4e09-a8e4-950f968c70d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:14.717503 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 736/1458] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:14.718525 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096357a2-a560-45b4-8558-8d2ed3491ba8 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:14.722251 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-096357a2-a560-45b4-8558-8d2ed3491ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:14.766531 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:14.825299 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096357a2-a560-45b4-8558-8d2ed3491ba8 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:14.830180 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e28753-75f4-4691-902c-b2454f984bf7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:14.830989 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e28753-75f4-4691-902c-b2454f984bf7 service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:14.831393 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e28753-75f4-4691-902c-b2454f984bf7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:14.831886 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e28753-75f4-4691-902c-b2454f984bf7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:14.887596 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 723/1459] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:14.888708 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442a859-1f86-4e09-a8e4-950f968c70d0 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:15.000877 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e28753-75f4-4691-902c-b2454f984bf7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:15.002968 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e28753-75f4-4691-902c-b2454f984bf7 service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:15.016852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 737/1460] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:14 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:15.026672 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442a859-1f86-4e09-a8e4-950f968c70d0 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:15.035685 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e28753-75f4-4691-902c-b2454f984bf7 service cinder] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:15.037409 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267ef01-34ea-4090-916f-db50c692ca43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:15.041249 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267ef01-34ea-4090-916f-db50c692ca43 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:15.041597 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267ef01-34ea-4090-916f-db50c692ca43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:15.041934 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267ef01-34ea-4090-916f-db50c692ca43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:15.042692 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1267ef01-34ea-4090-916f-db50c692ca43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:15.049700 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1267ef01-34ea-4090-916f-db50c692ca43 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:15.100264 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e28753-75f4-4691-902c-b2454f984bf7 service cinder] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:15.129311 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82940bfd-20be-4d87-b1e1-669c0195eae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:15.145633 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82940bfd-20be-4d87-b1e1-669c0195eae7 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:15.146916 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82940bfd-20be-4d87-b1e1-669c0195eae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:15.147340 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82940bfd-20be-4d87-b1e1-669c0195eae7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:15.149996 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82940bfd-20be-4d87-b1e1-669c0195eae7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:15.153519 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82940bfd-20be-4d87-b1e1-669c0195eae7 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:15.162346 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 724/1461] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:14 2022] POST /identity/v3/projects => generated 369 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:15.173722 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267ef01-34ea-4090-916f-db50c692ca43 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:15.230005 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 738/1462] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:15 2022] POST /identity/v3/projects => generated 365 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:15.231156 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82940bfd-20be-4d87-b1e1-669c0195eae7 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:15.310744 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267ef01-34ea-4090-916f-db50c692ca43 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:15.336106 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:15.374596 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82940bfd-20be-4d87-b1e1-669c0195eae7 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:15.380307 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c7073-467f-41a5-a5e5-406652a07e98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:15.380984 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c7073-467f-41a5-a5e5-406652a07e98 service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:15.381343 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c7073-467f-41a5-a5e5-406652a07e98 service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:15.381689 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c7073-467f-41a5-a5e5-406652a07e98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:15.384254 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-dd5c7073-467f-41a5-a5e5-406652a07e98 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:15.444587 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f8fef-3986-4177-8992-f4a7a79ffa95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:15.445305 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f8fef-3986-4177-8992-f4a7a79ffa95 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:15.446414 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f8fef-3986-4177-8992-f4a7a79ffa95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:15.446797 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f8fef-3986-4177-8992-f4a7a79ffa95 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:15.447585 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f8fef-3986-4177-8992-f4a7a79ffa95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:15.449612 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f8fef-3986-4177-8992-f4a7a79ffa95 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:15.512493 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 725/1463] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:15 2022] POST /identity/v3/users => generated 291 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:15.513482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f8fef-3986-4177-8992-f4a7a79ffa95 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:15.550371 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f8fef-3986-4177-8992-f4a7a79ffa95 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:15.562098 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf40faa-f408-4bf1-9ad4-fba03bab39fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:15.565222 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf40faa-f408-4bf1-9ad4-fba03bab39fc admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:15.566069 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf40faa-f408-4bf1-9ad4-fba03bab39fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:15.566069 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf40faa-f408-4bf1-9ad4-fba03bab39fc admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:15.566233 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5c7073-467f-41a5-a5e5-406652a07e98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:15.566422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf40faa-f408-4bf1-9ad4-fba03bab39fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:15.567437 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5c7073-467f-41a5-a5e5-406652a07e98 service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:15.574877 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf40faa-f408-4bf1-9ad4-fba03bab39fc admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:15.593947 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 739/1464] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:15.595554 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c7073-467f-41a5-a5e5-406652a07e98 service cinder] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:15.621921 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c7073-467f-41a5-a5e5-406652a07e98 service cinder] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:15.622406 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 726/1465] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:15 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:15.627747 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf40faa-f408-4bf1-9ad4-fba03bab39fc admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:15.641062 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a60f26-17b1-44a1-b19c-d2c080d20b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:15.641707 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a60f26-17b1-44a1-b19c-d2c080d20b63 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:15.644091 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf40faa-f408-4bf1-9ad4-fba03bab39fc admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:15.650132 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dd7bb7-355c-4cd7-9b98-7d194a3909e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:15.650965 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd7bb7-355c-4cd7-9b98-7d194a3909e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:15.651393 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd7bb7-355c-4cd7-9b98-7d194a3909e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:15.651972 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd7bb7-355c-4cd7-9b98-7d194a3909e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:15.652765 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dd7bb7-355c-4cd7-9b98-7d194a3909e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:15.654968 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dd7bb7-355c-4cd7-9b98-7d194a3909e4 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:15.656246 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a60f26-17b1-44a1-b19c-d2c080d20b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:15.656566 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a60f26-17b1-44a1-b19c-d2c080d20b63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:15.657650 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a60f26-17b1-44a1-b19c-d2c080d20b63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:15.662373 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a60f26-17b1-44a1-b19c-d2c080d20b63 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:15.668047 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:15.697204 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 740/1466] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:15.698816 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 727/1467] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:15.699410 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a60f26-17b1-44a1-b19c-d2c080d20b63 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:15.703604 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dd7bb7-355c-4cd7-9b98-7d194a3909e4 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:15.745074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a60f26-17b1-44a1-b19c-d2c080d20b63 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:15.746324 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dd7bb7-355c-4cd7-9b98-7d194a3909e4 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:15.767456 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5500f-c271-4c8b-8044-0488faed8ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:15.769950 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158a6af-018d-418b-b47c-1c15ec50e5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:15.771503 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5500f-c271-4c8b-8044-0488faed8ac2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:15.771873 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5500f-c271-4c8b-8044-0488faed8ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:15.772651 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158a6af-018d-418b-b47c-1c15ec50e5bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:15.773163 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5500f-c271-4c8b-8044-0488faed8ac2 admin admin] PATH_INFO: `/v3/projects/2578ef03784242c3843c6db2d6e87e82/users/6445d4cbef3a4726baa40b914c6f8d7d/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:15.774118 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158a6af-018d-418b-b47c-1c15ec50e5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:15.774432 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158a6af-018d-418b-b47c-1c15ec50e5bf admin admin] PATH_INFO: `/v3/projects/e87c2aa8fdba49b9bdc2b06b895c34ca/users/727f9b62f85747cc843cc5a90d5d0b01/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:15.832341 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5500f-c271-4c8b-8044-0488faed8ac2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2578ef03784242c3843c6db2d6e87e82, user_id=6445d4cbef3a4726baa40b914c6f8d7d, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:15.840609 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5500f-c271-4c8b-8044-0488faed8ac2 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:15.866796 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c158a6af-018d-418b-b47c-1c15ec50e5bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e87c2aa8fdba49b9bdc2b06b895c34ca, user_id=727f9b62f85747cc843cc5a90d5d0b01, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:15.877731 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c158a6af-018d-418b-b47c-1c15ec50e5bf admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:15.886419 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 741/1468] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:15 2022] PUT /identity/v3/projects/2578ef03784242c3843c6db2d6e87e82/users/6445d4cbef3a4726baa40b914c6f8d7d/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:15.890738 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5500f-c271-4c8b-8044-0488faed8ac2 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:15.922933 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 728/1469] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:15 2022] PUT /identity/v3/projects/e87c2aa8fdba49b9bdc2b06b895c34ca/users/727f9b62f85747cc843cc5a90d5d0b01/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:15.926906 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158a6af-018d-418b-b47c-1c15ec50e5bf admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:16.126019 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158a6af-018d-418b-b47c-1c15ec50e5bf admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:16.142840 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa133bd5-d11b-4261-85b5-75cbdf5e2605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:16.143639 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa133bd5-d11b-4261-85b5-75cbdf5e2605 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:16.146045 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa133bd5-d11b-4261-85b5-75cbdf5e2605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:16.146381 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa133bd5-d11b-4261-85b5-75cbdf5e2605 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:16.148608 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa133bd5-d11b-4261-85b5-75cbdf5e2605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:16.154994 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa133bd5-d11b-4261-85b5-75cbdf5e2605 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:16.166870 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5500f-c271-4c8b-8044-0488faed8ac2 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:16.179580 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e4b9c4-e15a-4e6d-88d6-c05a19d6e407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:16.180862 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4b9c4-e15a-4e6d-88d6-c05a19d6e407 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:16.181331 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4b9c4-e15a-4e6d-88d6-c05a19d6e407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:16.181602 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4b9c4-e15a-4e6d-88d6-c05a19d6e407 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:16.182259 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e4b9c4-e15a-4e6d-88d6-c05a19d6e407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:16.183970 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e4b9c4-e15a-4e6d-88d6-c05a19d6e407 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:16.190156 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 729/1470] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:16.192751 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa133bd5-d11b-4261-85b5-75cbdf5e2605 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:16.219437 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa133bd5-d11b-4261-85b5-75cbdf5e2605 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:16.236425 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434c380-3c5b-432c-9211-193d366f145b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:16.237158 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434c380-3c5b-432c-9211-193d366f145b admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:16.237434 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434c380-3c5b-432c-9211-193d366f145b admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:16.237752 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434c380-3c5b-432c-9211-193d366f145b admin admin] PATH_INFO: `/v3/projects/e87c2aa8fdba49b9bdc2b06b895c34ca/users/727f9b62f85747cc843cc5a90d5d0b01/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:16.241354 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434c380-3c5b-432c-9211-193d366f145b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e87c2aa8fdba49b9bdc2b06b895c34ca, user_id=727f9b62f85747cc843cc5a90d5d0b01, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:16.247456 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 742/1471] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:16.248923 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434c380-3c5b-432c-9211-193d366f145b admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:16.252716 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e4b9c4-e15a-4e6d-88d6-c05a19d6e407 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:16.287382 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 730/1472] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:16 2022] PUT /identity/v3/projects/e87c2aa8fdba49b9bdc2b06b895c34ca/users/727f9b62f85747cc843cc5a90d5d0b01/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:16.297406 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e4b9c4-e15a-4e6d-88d6-c05a19d6e407 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:16.299514 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920855ac-3400-416b-aa6f-a58353339b04 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:16.299904 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920855ac-3400-416b-aa6f-a58353339b04 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:16.300353 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920855ac-3400-416b-aa6f-a58353339b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:16.355477 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-920855ac-3400-416b-aa6f-a58353339b04 None None] MFA Rules not processed for user `727f9b62f85747cc843cc5a90d5d0b01`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:16.359463 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-920855ac-3400-416b-aa6f-a58353339b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:16.401447 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:16.540827 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff47a1-cf1f-4e29-aa60-f1effefc76b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:16.544895 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff47a1-cf1f-4e29-aa60-f1effefc76b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:16.548191 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff47a1-cf1f-4e29-aa60-f1effefc76b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:16.550961 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff47a1-cf1f-4e29-aa60-f1effefc76b3 admin admin] PATH_INFO: `/v3/projects/2578ef03784242c3843c6db2d6e87e82/users/6445d4cbef3a4726baa40b914c6f8d7d/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:16.566353 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff47a1-cf1f-4e29-aa60-f1effefc76b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2578ef03784242c3843c6db2d6e87e82, user_id=6445d4cbef3a4726baa40b914c6f8d7d, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:16.577103 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff47a1-cf1f-4e29-aa60-f1effefc76b3 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:16.598282 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 731/1473] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:16 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:16.615276 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920855ac-3400-416b-aa6f-a58353339b04 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:16.629229 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 743/1474] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:16 2022] PUT /identity/v3/projects/2578ef03784242c3843c6db2d6e87e82/users/6445d4cbef3a4726baa40b914c6f8d7d/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:16.641453 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5141c5-b1a5-49d0-9aa8-99fd168388df None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:16.641813 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5141c5-b1a5-49d0-9aa8-99fd168388df None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:16.642027 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5141c5-b1a5-49d0-9aa8-99fd168388df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:16.719431 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-3b5141c5-b1a5-49d0-9aa8-99fd168388df None None] MFA Rules not processed for user `6445d4cbef3a4726baa40b914c6f8d7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:16.720777 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-3b5141c5-b1a5-49d0-9aa8-99fd168388df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:16.734684 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920855ac-3400-416b-aa6f-a58353339b04 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:16.807837 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:16.827979 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2c015-6ee8-4c93-a1c2-77c244ec8cd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:16.829004 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2c015-6ee8-4c93-a1c2-77c244ec8cd5 service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:16.829706 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2c015-6ee8-4c93-a1c2-77c244ec8cd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:16.830304 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2c015-6ee8-4c93-a1c2-77c244ec8cd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:16.841904 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 744/1475] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:16 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:16.853219 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5141c5-b1a5-49d0-9aa8-99fd168388df None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:16.856268 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-3b5141c5-b1a5-49d0-9aa8-99fd168388df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:16.877132 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5141c5-b1a5-49d0-9aa8-99fd168388df None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:16.885460 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b87cf-2b62-4ae4-9352-25bc7ccba4c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:16.887319 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b87cf-2b62-4ae4-9352-25bc7ccba4c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:16.887599 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b87cf-2b62-4ae4-9352-25bc7ccba4c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:16.887873 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b87cf-2b62-4ae4-9352-25bc7ccba4c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:16.909322 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b87cf-2b62-4ae4-9352-25bc7ccba4c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:16.912505 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b87cf-2b62-4ae4-9352-25bc7ccba4c6 service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:16.949626 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 745/1476] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:16 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:16.980353 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f2c015-6ee8-4c93-a1c2-77c244ec8cd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:16.981891 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f2c015-6ee8-4c93-a1c2-77c244ec8cd5 service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:17.007608 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 732/1477] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:16 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:28:17.217206 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce4173-488a-4895-a2fa-58be279499e3 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:17.217496 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce4173-488a-4895-a2fa-58be279499e3 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:17.217846 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce4173-488a-4895-a2fa-58be279499e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:17.254638 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2c015-6ee8-4c93-a1c2-77c244ec8cd5 service cinder] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:17.270008 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2c015-6ee8-4c93-a1c2-77c244ec8cd5 service cinder] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:17.274632 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-98ce4173-488a-4895-a2fa-58be279499e3 None None] MFA Rules not processed for user `be586640013949589d2a7b11a7a2f3b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:17.275905 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-98ce4173-488a-4895-a2fa-58be279499e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:17.283448 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be42f7-9371-4c47-b4cd-32089a6a8a82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:17.284013 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be42f7-9371-4c47-b4cd-32089a6a8a82 service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:17.284610 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be42f7-9371-4c47-b4cd-32089a6a8a82 service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:17.284610 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be42f7-9371-4c47-b4cd-32089a6a8a82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:17.369559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:17.418335 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 746/1478] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:17.492500 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58be42f7-9371-4c47-b4cd-32089a6a8a82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:17.494022 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58be42f7-9371-4c47-b4cd-32089a6a8a82 service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:17.524964 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 733/1479] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:17.916170 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ce4173-488a-4895-a2fa-58be279499e3 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:17.919157 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-98ce4173-488a-4895-a2fa-58be279499e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:18.140757 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ce4173-488a-4895-a2fa-58be279499e3 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:18.145540 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3fc04f-2a1b-4d44-b26e-64e015c0f495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:18.146289 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3fc04f-2a1b-4d44-b26e-64e015c0f495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:18.147102 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3fc04f-2a1b-4d44-b26e-64e015c0f495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:18.147102 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3fc04f-2a1b-4d44-b26e-64e015c0f495 admin admin] PATH_INFO: `/v3/users/3f3fd7a3a86c48d1925d00edcb2caec2` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:18.150245 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3fc04f-2a1b-4d44-b26e-64e015c0f495 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f3fd7a3a86c48d1925d00edcb2caec2)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:18.152449 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3fc04f-2a1b-4d44-b26e-64e015c0f495 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:18.300396 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5a3fc04f-2a1b-4d44-b26e-64e015c0f495 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f3fd7a3a86c48d1925d00edcb2caec2'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:18.309713 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5a3fc04f-2a1b-4d44-b26e-64e015c0f495 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f3fd7a3a86c48d1925d00edcb2caec2'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:18.310548 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5a3fc04f-2a1b-4d44-b26e-64e015c0f495 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f3fd7a3a86c48d1925d00edcb2caec2'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:18.344119 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5a3fc04f-2a1b-4d44-b26e-64e015c0f495 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f3fd7a3a86c48d1925d00edcb2caec2'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:18.368033 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5a3fc04f-2a1b-4d44-b26e-64e015c0f495 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f3fd7a3a86c48d1925d00edcb2caec2'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:18.372043 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:18.392429 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 747/1480] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:17 2022] DELETE /identity/v3/users/3f3fd7a3a86c48d1925d00edcb2caec2 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:18.396568 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f11923-6a9d-469b-b609-4345ef02790a None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:18.396568 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f11923-6a9d-469b-b609-4345ef02790a None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:18.396568 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f11923-6a9d-469b-b609-4345ef02790a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:18.425221 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af88af7-c374-44c8-b47a-11583bb540ed None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:18.425657 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af88af7-c374-44c8-b47a-11583bb540ed None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:18.426139 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af88af7-c374-44c8-b47a-11583bb540ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:18.461210 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-b3f11923-6a9d-469b-b609-4345ef02790a None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:18.462461 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-b3f11923-6a9d-469b-b609-4345ef02790a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:18.584050 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-0af88af7-c374-44c8-b47a-11583bb540ed None None] MFA Rules not processed for user `364555ec385c4dbb91907fb9c5116d2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:18.588006 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-0af88af7-c374-44c8-b47a-11583bb540ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:18.730375 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 734/1481] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:18.834008 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 748/1482] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Nov 2 07:28:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:18.870216 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f11923-6a9d-469b-b609-4345ef02790a None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:18.872379 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-b3f11923-6a9d-469b-b609-4345ef02790a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:19.111872 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af88af7-c374-44c8-b47a-11583bb540ed None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:19.113310 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-0af88af7-c374-44c8-b47a-11583bb540ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:19.149909 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f11923-6a9d-469b-b609-4345ef02790a None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:19.163027 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3050c58-775d-4005-ab4b-ab80e4eaf83d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:19.165199 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3050c58-775d-4005-ab4b-ab80e4eaf83d service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:19.167834 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3050c58-775d-4005-ab4b-ab80e4eaf83d service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:19.168318 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3050c58-775d-4005-ab4b-ab80e4eaf83d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:19.191532 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3050c58-775d-4005-ab4b-ab80e4eaf83d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:19.209262 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3050c58-775d-4005-ab4b-ab80e4eaf83d service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:19.261292 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 735/1483] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:19.411754 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af88af7-c374-44c8-b47a-11583bb540ed None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:19.417364 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2469fc-a7b5-480a-b6a5-8d182c90d2c7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '836c4044aaf74de391219289b57d2956', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:19.417990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2469fc-a7b5-480a-b6a5-8d182c90d2c7 service swift] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:19.418261 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2469fc-a7b5-480a-b6a5-8d182c90d2c7 service swift] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:19.418527 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2469fc-a7b5-480a-b6a5-8d182c90d2c7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:19.431984 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2469fc-a7b5-480a-b6a5-8d182c90d2c7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:19.433654 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2469fc-a7b5-480a-b6a5-8d182c90d2c7 service swift] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:19.465023 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 749/1484] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Nov 2 07:28:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Nov 02 07:28:20.479906 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3050c58-775d-4005-ab4b-ab80e4eaf83d service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:20.482131 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-b3050c58-775d-4005-ab4b-ab80e4eaf83d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:20.509134 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3050c58-775d-4005-ab4b-ab80e4eaf83d service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:20.514470 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0dfaa6-d230-4941-b068-01640a29ae88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:20.515166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0dfaa6-d230-4941-b068-01640a29ae88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:20.515449 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0dfaa6-d230-4941-b068-01640a29ae88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:20.515715 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0dfaa6-d230-4941-b068-01640a29ae88 admin admin] PATH_INFO: `/v3/projects/54199d35d6944698be0cb633ad56c7e5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:20.516944 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0dfaa6-d230-4941-b068-01640a29ae88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54199d35d6944698be0cb633ad56c7e5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:20.520962 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0dfaa6-d230-4941-b068-01640a29ae88 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:20.574985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-bc0dfaa6-d230-4941-b068-01640a29ae88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54199d35d6944698be0cb633ad56c7e5', 'request_id': 'req-bc0dfaa6-d230-4941-b068-01640a29ae88'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:20.581756 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-bc0dfaa6-d230-4941-b068-01640a29ae88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54199d35d6944698be0cb633ad56c7e5', 'request_id': 'req-bc0dfaa6-d230-4941-b068-01640a29ae88'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:20.590047 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-bc0dfaa6-d230-4941-b068-01640a29ae88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54199d35d6944698be0cb633ad56c7e5', 'request_id': 'req-bc0dfaa6-d230-4941-b068-01640a29ae88'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:20.600134 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:20.620207 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-bc0dfaa6-d230-4941-b068-01640a29ae88 admin admin] The token cache is being invalidate because project 54199d35d6944698be0cb633ad56c7e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:20.620645 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-bc0dfaa6-d230-4941-b068-01640a29ae88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:20.621477 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-bc0dfaa6-d230-4941-b068-01640a29ae88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:20.623170 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 736/1485] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:20 2022] DELETE /identity/v3/projects/54199d35d6944698be0cb633ad56c7e5 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:20.628387 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c926a-4818-4779-8b52-b8640bbe06f8 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:20.628908 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c926a-4818-4779-8b52-b8640bbe06f8 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:20.628996 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c926a-4818-4779-8b52-b8640bbe06f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:20.650641 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-e71c926a-4818-4779-8b52-b8640bbe06f8 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:20.651892 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-e71c926a-4818-4779-8b52-b8640bbe06f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:20.696533 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:20.723030 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 750/1486] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:20.732066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2826e-a0d8-43e7-bf3c-86e6d7430c02 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:20.732543 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2826e-a0d8-43e7-bf3c-86e6d7430c02 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:20.732950 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2826e-a0d8-43e7-bf3c-86e6d7430c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:20.761171 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-dec2826e-a0d8-43e7-bf3c-86e6d7430c02 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:20.762681 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-dec2826e-a0d8-43e7-bf3c-86e6d7430c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:20.793907 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 737/1487] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:20.796322 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c926a-4818-4779-8b52-b8640bbe06f8 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:20.809915 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c926a-4818-4779-8b52-b8640bbe06f8 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:20.816384 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2835ba03-0c8f-419c-8a5e-122bc2d34565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:20.817013 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2835ba03-0c8f-419c-8a5e-122bc2d34565 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:20.817426 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2835ba03-0c8f-419c-8a5e-122bc2d34565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:20.818064 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2835ba03-0c8f-419c-8a5e-122bc2d34565 admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:20.819682 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2835ba03-0c8f-419c-8a5e-122bc2d34565 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:20.821573 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2835ba03-0c8f-419c-8a5e-122bc2d34565 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:20.825092 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2835ba03-0c8f-419c-8a5e-122bc2d34565 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:20.840710 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 751/1488] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:28:20.845106 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec2826e-a0d8-43e7-bf3c-86e6d7430c02 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:20.862441 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec2826e-a0d8-43e7-bf3c-86e6d7430c02 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:20.871684 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7e2a0-909d-4a7d-bb10-3e4cc0df335e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:20.872384 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7e2a0-909d-4a7d-bb10-3e4cc0df335e admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:20.872790 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7e2a0-909d-4a7d-bb10-3e4cc0df335e admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:20.873067 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7e2a0-909d-4a7d-bb10-3e4cc0df335e admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:20.874018 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce7e2a0-909d-4a7d-bb10-3e4cc0df335e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:20.875954 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce7e2a0-909d-4a7d-bb10-3e4cc0df335e admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:20.907908 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 738/1489] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:20 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:20.912240 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2835ba03-0c8f-419c-8a5e-122bc2d34565 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:20.924352 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2835ba03-0c8f-419c-8a5e-122bc2d34565 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:21.011025 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d2580-8c60-4de2-aac1-c1f6af84be14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:21.011649 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d2580-8c60-4de2-aac1-c1f6af84be14 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:21.011926 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d2580-8c60-4de2-aac1-c1f6af84be14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:21.012195 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d2580-8c60-4de2-aac1-c1f6af84be14 admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:21.012832 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0d2580-8c60-4de2-aac1-c1f6af84be14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:21.014884 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0d2580-8c60-4de2-aac1-c1f6af84be14 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:21.039476 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 752/1490] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:20 2022] POST /identity/v3/users => generated 293 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:28:21.043880 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7e2a0-909d-4a7d-bb10-3e4cc0df335e admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:21.056625 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7e2a0-909d-4a7d-bb10-3e4cc0df335e admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:21.062859 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf6272-7d90-4660-ba93-bf6a064b4dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:21.063745 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf6272-7d90-4660-ba93-bf6a064b4dff admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:21.064218 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf6272-7d90-4660-ba93-bf6a064b4dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:21.064492 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf6272-7d90-4660-ba93-bf6a064b4dff admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:21.065420 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf6272-7d90-4660-ba93-bf6a064b4dff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:21.067291 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf6272-7d90-4660-ba93-bf6a064b4dff admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:21.079757 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 739/1491] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:21.083958 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d2580-8c60-4de2-aac1-c1f6af84be14 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:21.096589 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d2580-8c60-4de2-aac1-c1f6af84be14 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:21.109614 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503e663-b6d7-4814-b083-31740ba83d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:21.110282 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503e663-b6d7-4814-b083-31740ba83d0e admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:21.110563 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503e663-b6d7-4814-b083-31740ba83d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:21.110866 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503e663-b6d7-4814-b083-31740ba83d0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:21.111517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c503e663-b6d7-4814-b083-31740ba83d0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:21.113136 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c503e663-b6d7-4814-b083-31740ba83d0e admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:21.125753 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf6272-7d90-4660-ba93-bf6a064b4dff admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:21.130218 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-97bf6272-7d90-4660-ba93-bf6a064b4dff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:21.132582 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 753/1492] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:21.136927 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503e663-b6d7-4814-b083-31740ba83d0e admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:21.145894 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf6272-7d90-4660-ba93-bf6a064b4dff admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:21.146807 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503e663-b6d7-4814-b083-31740ba83d0e admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:21.150676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3855c12e-0264-4af6-803b-678e80432605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:21.151314 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3855c12e-0264-4af6-803b-678e80432605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:21.151489 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61be360-df03-4f64-b2cd-d0eb916886f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:21.151600 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3855c12e-0264-4af6-803b-678e80432605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:21.151834 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3855c12e-0264-4af6-803b-678e80432605 admin admin] PATH_INFO: `/v3/users/b1374a56b69c4ccfa0cc92ee3b53493a` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:21.152126 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61be360-df03-4f64-b2cd-d0eb916886f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:21.152398 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61be360-df03-4f64-b2cd-d0eb916886f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:21.152661 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61be360-df03-4f64-b2cd-d0eb916886f0 admin admin] PATH_INFO: `/v3/projects/2d30a300103c41a7a2e3a634c59d2a48/users/957c2507c03846208b79b8c262384d1d/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:21.152950 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3855c12e-0264-4af6-803b-678e80432605 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1374a56b69c4ccfa0cc92ee3b53493a)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:21.155308 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3855c12e-0264-4af6-803b-678e80432605 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:21.170176 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61be360-df03-4f64-b2cd-d0eb916886f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d30a300103c41a7a2e3a634c59d2a48, user_id=957c2507c03846208b79b8c262384d1d, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:21.173744 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61be360-df03-4f64-b2cd-d0eb916886f0 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:21.194960 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 754/1493] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:21 2022] PUT /identity/v3/projects/2d30a300103c41a7a2e3a634c59d2a48/users/957c2507c03846208b79b8c262384d1d/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:21.208531 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84bb86d-4a54-4c51-8f19-d99cde5ec1e5 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:21.208657 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84bb86d-4a54-4c51-8f19-d99cde5ec1e5 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:21.208932 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84bb86d-4a54-4c51-8f19-d99cde5ec1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:21.222904 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-3855c12e-0264-4af6-803b-678e80432605 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1374a56b69c4ccfa0cc92ee3b53493a'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:21.223644 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-3855c12e-0264-4af6-803b-678e80432605 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1374a56b69c4ccfa0cc92ee3b53493a'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:21.232219 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-3855c12e-0264-4af6-803b-678e80432605 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1374a56b69c4ccfa0cc92ee3b53493a'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:21.241614 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-b84bb86d-4a54-4c51-8f19-d99cde5ec1e5 None None] MFA Rules not processed for user `957c2507c03846208b79b8c262384d1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:21.243042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-b84bb86d-4a54-4c51-8f19-d99cde5ec1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:21.246433 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-3855c12e-0264-4af6-803b-678e80432605 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1374a56b69c4ccfa0cc92ee3b53493a'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:21.258402 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-3855c12e-0264-4af6-803b-678e80432605 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1374a56b69c4ccfa0cc92ee3b53493a'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:21.266932 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 740/1494] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:21 2022] DELETE /identity/v3/users/b1374a56b69c4ccfa0cc92ee3b53493a => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:21.271862 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca00c4-d0b0-4323-9e2b-279ace2438a2 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:21.272147 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca00c4-d0b0-4323-9e2b-279ace2438a2 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:21.272442 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca00c4-d0b0-4323-9e2b-279ace2438a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:21.292772 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-36ca00c4-d0b0-4323-9e2b-279ace2438a2 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:21.294149 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-36ca00c4-d0b0-4323-9e2b-279ace2438a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:21.314636 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 755/1495] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:21.319884 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84bb86d-4a54-4c51-8f19-d99cde5ec1e5 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:21.321591 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-b84bb86d-4a54-4c51-8f19-d99cde5ec1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:21.335541 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:21.356336 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 741/1496] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:21.366576 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca00c4-d0b0-4323-9e2b-279ace2438a2 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:21.368151 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-36ca00c4-d0b0-4323-9e2b-279ace2438a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:21.393227 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84bb86d-4a54-4c51-8f19-d99cde5ec1e5 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:21.397900 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43153a-6704-49de-86b5-1ce994815590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:21.398639 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43153a-6704-49de-86b5-1ce994815590 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:21.399142 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43153a-6704-49de-86b5-1ce994815590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:21.399410 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43153a-6704-49de-86b5-1ce994815590 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:21.399844 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:21.400040 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a43153a-6704-49de-86b5-1ce994815590 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:21.402796 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a43153a-6704-49de-86b5-1ce994815590 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:21.433611 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca00c4-d0b0-4323-9e2b-279ace2438a2 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:21.437927 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b86fc3-5d61-45c7-a831-38bcac556be1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:21.438631 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b86fc3-5d61-45c7-a831-38bcac556be1 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:21.439236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b86fc3-5d61-45c7-a831-38bcac556be1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:21.439236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b86fc3-5d61-45c7-a831-38bcac556be1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:21.440315 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 756/1497] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:21 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:28:21.444616 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43153a-6704-49de-86b5-1ce994815590 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:21.450079 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b86fc3-5d61-45c7-a831-38bcac556be1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:21.451610 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b86fc3-5d61-45c7-a831-38bcac556be1 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:21.465965 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 742/1498] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:28:21.533400 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43153a-6704-49de-86b5-1ce994815590 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:21.538994 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f8a4f-646a-43e3-8332-99552c94ac12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:21.539619 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f8a4f-646a-43e3-8332-99552c94ac12 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:21.540204 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f8a4f-646a-43e3-8332-99552c94ac12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:21.540382 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f8a4f-646a-43e3-8332-99552c94ac12 admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:21.541030 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77f8a4f-646a-43e3-8332-99552c94ac12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:21.543137 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77f8a4f-646a-43e3-8332-99552c94ac12 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:21.567972 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 757/1499] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:21 2022] POST /identity/v3/users => generated 292 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:21.572146 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b86fc3-5d61-45c7-a831-38bcac556be1 service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:21.582000 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b86fc3-5d61-45c7-a831-38bcac556be1 service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:21.587164 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32f696-ef0f-4456-9d7c-1d4ae211f7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:21.587774 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32f696-ef0f-4456-9d7c-1d4ae211f7f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:21.588045 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32f696-ef0f-4456-9d7c-1d4ae211f7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:21.588309 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32f696-ef0f-4456-9d7c-1d4ae211f7f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:21.588971 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad32f696-ef0f-4456-9d7c-1d4ae211f7f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:21.590561 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad32f696-ef0f-4456-9d7c-1d4ae211f7f5 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:21.604594 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 743/1500] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:28:21.608817 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f8a4f-646a-43e3-8332-99552c94ac12 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:21.619585 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f8a4f-646a-43e3-8332-99552c94ac12 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:21.625102 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cf4611-475a-4b52-b650-a9395992ba90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:21.625719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf4611-475a-4b52-b650-a9395992ba90 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:21.626352 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf4611-475a-4b52-b650-a9395992ba90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:21.626352 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf4611-475a-4b52-b650-a9395992ba90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:21.627248 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cf4611-475a-4b52-b650-a9395992ba90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:21.628854 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cf4611-475a-4b52-b650-a9395992ba90 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:21.640754 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 758/1501] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:21.645233 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32f696-ef0f-4456-9d7c-1d4ae211f7f5 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:21.656187 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32f696-ef0f-4456-9d7c-1d4ae211f7f5 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:21.661369 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba370f-9b70-4d77-be09-5a7c49e87c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:21.662030 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba370f-9b70-4d77-be09-5a7c49e87c19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:21.662306 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba370f-9b70-4d77-be09-5a7c49e87c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:21.662581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba370f-9b70-4d77-be09-5a7c49e87c19 admin admin] PATH_INFO: `/v3/projects/52940055c035479583438fac39e541b2/users/65612a2e487348eea96254a3b6132ce0/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:21.678831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba370f-9b70-4d77-be09-5a7c49e87c19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52940055c035479583438fac39e541b2, user_id=65612a2e487348eea96254a3b6132ce0, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:21.682089 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba370f-9b70-4d77-be09-5a7c49e87c19 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:21.697880 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 744/1502] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:21 2022] PUT /identity/v3/projects/52940055c035479583438fac39e541b2/users/65612a2e487348eea96254a3b6132ce0/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:21.708123 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a99ab-9811-4f16-8377-eaa398d8f93a None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:21.708408 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a99ab-9811-4f16-8377-eaa398d8f93a None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:21.708680 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a99ab-9811-4f16-8377-eaa398d8f93a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:21.765257 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-8a3a99ab-9811-4f16-8377-eaa398d8f93a None None] MFA Rules not processed for user `65612a2e487348eea96254a3b6132ce0`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:21.767129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-8a3a99ab-9811-4f16-8377-eaa398d8f93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:21.792390 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba370f-9b70-4d77-be09-5a7c49e87c19 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:21.794685 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-2bba370f-9b70-4d77-be09-5a7c49e87c19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:21.898965 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 759/1503] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:21.903103 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a99ab-9811-4f16-8377-eaa398d8f93a None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:21.914545 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:22.023166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba370f-9b70-4d77-be09-5a7c49e87c19 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:22.037531 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecccb2b9-b5a7-4d98-bfa4-56f9ddca7b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:22.038144 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecccb2b9-b5a7-4d98-bfa4-56f9ddca7b03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:22.039095 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecccb2b9-b5a7-4d98-bfa4-56f9ddca7b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:22.039095 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecccb2b9-b5a7-4d98-bfa4-56f9ddca7b03 admin admin] PATH_INFO: `/v3/projects/ddfd598da25f41269dcd457b1120d08a` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:22.043088 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecccb2b9-b5a7-4d98-bfa4-56f9ddca7b03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddfd598da25f41269dcd457b1120d08a)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:22.045077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecccb2b9-b5a7-4d98-bfa4-56f9ddca7b03 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:22.053870 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a99ab-9811-4f16-8377-eaa398d8f93a None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:22.062500 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e2cea-b5f6-4050-8fa9-1334af3822e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:22.063406 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e2cea-b5f6-4050-8fa9-1334af3822e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:22.063675 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e2cea-b5f6-4050-8fa9-1334af3822e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:22.063943 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e2cea-b5f6-4050-8fa9-1334af3822e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:22.064648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089e2cea-b5f6-4050-8fa9-1334af3822e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:22.067107 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089e2cea-b5f6-4050-8fa9-1334af3822e9 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:22.122803 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 760/1504] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:21 2022] POST /identity/v3/projects => generated 383 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:28:22.123676 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e2cea-b5f6-4050-8fa9-1334af3822e9 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:22.126040 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-089e2cea-b5f6-4050-8fa9-1334af3822e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:22.180431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-ecccb2b9-b5a7-4d98-bfa4-56f9ddca7b03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddfd598da25f41269dcd457b1120d08a', 'request_id': 'req-ecccb2b9-b5a7-4d98-bfa4-56f9ddca7b03'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:22.190809 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-ecccb2b9-b5a7-4d98-bfa4-56f9ddca7b03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddfd598da25f41269dcd457b1120d08a', 'request_id': 'req-ecccb2b9-b5a7-4d98-bfa4-56f9ddca7b03'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:22.210561 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-ecccb2b9-b5a7-4d98-bfa4-56f9ddca7b03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddfd598da25f41269dcd457b1120d08a', 'request_id': 'req-ecccb2b9-b5a7-4d98-bfa4-56f9ddca7b03'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:22.235588 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e2cea-b5f6-4050-8fa9-1334af3822e9 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:22.237739 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-ecccb2b9-b5a7-4d98-bfa4-56f9ddca7b03 admin admin] The token cache is being invalidate because project ddfd598da25f41269dcd457b1120d08a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:22.238110 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-ecccb2b9-b5a7-4d98-bfa4-56f9ddca7b03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:22.238880 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-ecccb2b9-b5a7-4d98-bfa4-56f9ddca7b03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:22.240856 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 745/1505] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:21 2022] DELETE /identity/v3/projects/ddfd598da25f41269dcd457b1120d08a => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:22.243611 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecccb2b9-b5a7-4d98-bfa4-56f9ddca7b03 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:22.245487 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-ecccb2b9-b5a7-4d98-bfa4-56f9ddca7b03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:22.305905 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:22.311810 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e5d9b4-6e0c-4c32-8f7b-d18c80516916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:22.312493 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5d9b4-6e0c-4c32-8f7b-d18c80516916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:22.312944 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5d9b4-6e0c-4c32-8f7b-d18c80516916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:22.313302 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5d9b4-6e0c-4c32-8f7b-d18c80516916 admin admin] PATH_INFO: `/v3/users/7538b9215f2848408b9b9282dccebd34` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:22.314721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e5d9b4-6e0c-4c32-8f7b-d18c80516916 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7538b9215f2848408b9b9282dccebd34)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:22.316948 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e5d9b4-6e0c-4c32-8f7b-d18c80516916 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:22.368678 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecccb2b9-b5a7-4d98-bfa4-56f9ddca7b03 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:22.374273 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00767ca7-adaa-41eb-a11a-e5d91d6434e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:22.374959 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00767ca7-adaa-41eb-a11a-e5d91d6434e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:22.375231 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00767ca7-adaa-41eb-a11a-e5d91d6434e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:22.381151 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00767ca7-adaa-41eb-a11a-e5d91d6434e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:22.382084 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00767ca7-adaa-41eb-a11a-e5d91d6434e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:22.386448 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00767ca7-adaa-41eb-a11a-e5d91d6434e9 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:22.394179 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:22.402528 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-44e5d9b4-6e0c-4c32-8f7b-d18c80516916 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7538b9215f2848408b9b9282dccebd34'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:22.421801 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 746/1506] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:22 2022] POST /identity/v3/users => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:22.422168 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-44e5d9b4-6e0c-4c32-8f7b-d18c80516916 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7538b9215f2848408b9b9282dccebd34'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:22.422168 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-44e5d9b4-6e0c-4c32-8f7b-d18c80516916 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7538b9215f2848408b9b9282dccebd34'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:22.422322 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00767ca7-adaa-41eb-a11a-e5d91d6434e9 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:22.424081 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-00767ca7-adaa-41eb-a11a-e5d91d6434e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:22.438610 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-44e5d9b4-6e0c-4c32-8f7b-d18c80516916 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7538b9215f2848408b9b9282dccebd34'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:22.443536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00767ca7-adaa-41eb-a11a-e5d91d6434e9 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:22.448623 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-44e5d9b4-6e0c-4c32-8f7b-d18c80516916 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7538b9215f2848408b9b9282dccebd34'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:22.455445 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4bf58-9c89-4926-ab92-48f48854c057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:22.456701 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4bf58-9c89-4926-ab92-48f48854c057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:22.457361 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4bf58-9c89-4926-ab92-48f48854c057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:22.457918 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4bf58-9c89-4926-ab92-48f48854c057 admin admin] PATH_INFO: `/v3/users/6445d4cbef3a4726baa40b914c6f8d7d` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:22.459825 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c4bf58-9c89-4926-ab92-48f48854c057 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6445d4cbef3a4726baa40b914c6f8d7d)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:22.460088 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 761/1507] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:22 2022] DELETE /identity/v3/users/7538b9215f2848408b9b9282dccebd34 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:22.461288 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e5d9b4-6e0c-4c32-8f7b-d18c80516916 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:22.462989 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-44e5d9b4-6e0c-4c32-8f7b-d18c80516916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:22.464775 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c4bf58-9c89-4926-ab92-48f48854c057 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:22.526919 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-61c4bf58-9c89-4926-ab92-48f48854c057 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6445d4cbef3a4726baa40b914c6f8d7d'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:22.528062 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-61c4bf58-9c89-4926-ab92-48f48854c057 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6445d4cbef3a4726baa40b914c6f8d7d'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:22.537018 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-61c4bf58-9c89-4926-ab92-48f48854c057 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6445d4cbef3a4726baa40b914c6f8d7d'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:22.545247 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e5d9b4-6e0c-4c32-8f7b-d18c80516916 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:22.552197 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267f6e5-1733-4333-b424-826fcf3733aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:22.552317 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-61c4bf58-9c89-4926-ab92-48f48854c057 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6445d4cbef3a4726baa40b914c6f8d7d'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:22.552672 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267f6e5-1733-4333-b424-826fcf3733aa admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:22.552946 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267f6e5-1733-4333-b424-826fcf3733aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:22.553413 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267f6e5-1733-4333-b424-826fcf3733aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:22.554592 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1267f6e5-1733-4333-b424-826fcf3733aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:22.557362 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1267f6e5-1733-4333-b424-826fcf3733aa admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:22.574486 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 762/1508] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:22.579112 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4429b54-477b-4134-b37e-adef549c64f4 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:22.579506 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4429b54-477b-4134-b37e-adef549c64f4 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:22.579680 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4429b54-477b-4134-b37e-adef549c64f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:22.581073 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-61c4bf58-9c89-4926-ab92-48f48854c057 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6445d4cbef3a4726baa40b914c6f8d7d'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:22.595996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 747/1509] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:22 2022] DELETE /identity/v3/users/6445d4cbef3a4726baa40b914c6f8d7d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:22.598442 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4bf58-9c89-4926-ab92-48f48854c057 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:22.605265 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-61c4bf58-9c89-4926-ab92-48f48854c057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:22.617974 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-a4429b54-477b-4134-b37e-adef549c64f4 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:22.619857 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-a4429b54-477b-4134-b37e-adef549c64f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:22.706210 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4bf58-9c89-4926-ab92-48f48854c057 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:22.712629 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfddc2e-2637-42a6-8112-5bc4be810332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:22.713648 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfddc2e-2637-42a6-8112-5bc4be810332 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:22.714133 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfddc2e-2637-42a6-8112-5bc4be810332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:22.714676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfddc2e-2637-42a6-8112-5bc4be810332 admin admin] PATH_INFO: `/v3/projects/d5d3495a3b0e4b91bc389964ef808847/users/47082dff733d4dd68dddf1dbd435f084/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:22.716080 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 763/1510] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:22.747443 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4429b54-477b-4134-b37e-adef549c64f4 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:22.748492 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfddc2e-2637-42a6-8112-5bc4be810332 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5d3495a3b0e4b91bc389964ef808847, user_id=47082dff733d4dd68dddf1dbd435f084, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:22.753713 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-a4429b54-477b-4134-b37e-adef549c64f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:22.755198 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfddc2e-2637-42a6-8112-5bc4be810332 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:22.815829 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 748/1511] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:22 2022] PUT /identity/v3/projects/d5d3495a3b0e4b91bc389964ef808847/users/47082dff733d4dd68dddf1dbd435f084/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:22.820470 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfddc2e-2637-42a6-8112-5bc4be810332 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:22.841280 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:22.868700 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4429b54-477b-4134-b37e-adef549c64f4 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:22.879686 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:22.974858 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfddc2e-2637-42a6-8112-5bc4be810332 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:22.993806 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb83e2-85dd-4ef7-b56d-39bb6dae0f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:22.994704 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb83e2-85dd-4ef7-b56d-39bb6dae0f5a admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:22.995151 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb83e2-85dd-4ef7-b56d-39bb6dae0f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:22.995442 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb83e2-85dd-4ef7-b56d-39bb6dae0f5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:22.996472 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bb83e2-85dd-4ef7-b56d-39bb6dae0f5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:23.000562 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bb83e2-85dd-4ef7-b56d-39bb6dae0f5a admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:23.017157 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea17f8-d1c2-407c-99f6-a6e14fc30a80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:23.019622 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea17f8-d1c2-407c-99f6-a6e14fc30a80 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:23.021124 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea17f8-d1c2-407c-99f6-a6e14fc30a80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:23.023037 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea17f8-d1c2-407c-99f6-a6e14fc30a80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:23.042302 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 749/1512] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:28:23.044115 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb83e2-85dd-4ef7-b56d-39bb6dae0f5a admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:23.052528 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ea17f8-d1c2-407c-99f6-a6e14fc30a80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:23.057718 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ea17f8-d1c2-407c-99f6-a6e14fc30a80 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:23.064330 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb83e2-85dd-4ef7-b56d-39bb6dae0f5a admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:23.079698 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842ea814-f6fd-4ea7-99e0-c205b1e28ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:23.081136 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ea814-f6fd-4ea7-99e0-c205b1e28ac9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:23.081545 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ea814-f6fd-4ea7-99e0-c205b1e28ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:23.081846 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ea814-f6fd-4ea7-99e0-c205b1e28ac9 admin admin] PATH_INFO: `/v3/projects/d5d3495a3b0e4b91bc389964ef808847/users/47082dff733d4dd68dddf1dbd435f084/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:23.091823 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 764/1513] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:23.105393 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842ea814-f6fd-4ea7-99e0-c205b1e28ac9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5d3495a3b0e4b91bc389964ef808847, user_id=47082dff733d4dd68dddf1dbd435f084, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:23.108656 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842ea814-f6fd-4ea7-99e0-c205b1e28ac9 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:23.123359 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 750/1514] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:23 2022] PUT /identity/v3/projects/d5d3495a3b0e4b91bc389964ef808847/users/47082dff733d4dd68dddf1dbd435f084/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:23.136170 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866cba8f-182b-443b-a3f4-765bad445588 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:23.136560 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866cba8f-182b-443b-a3f4-765bad445588 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:23.136782 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866cba8f-182b-443b-a3f4-765bad445588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:23.199680 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-866cba8f-182b-443b-a3f4-765bad445588 None None] MFA Rules not processed for user `47082dff733d4dd68dddf1dbd435f084`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:23.204240 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-866cba8f-182b-443b-a3f4-765bad445588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:23.356049 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 765/1515] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:23 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:23.370205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842ea814-f6fd-4ea7-99e0-c205b1e28ac9 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:23.372911 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-842ea814-f6fd-4ea7-99e0-c205b1e28ac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:23.456653 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866cba8f-182b-443b-a3f4-765bad445588 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:23.458155 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-866cba8f-182b-443b-a3f4-765bad445588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:23.465812 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842ea814-f6fd-4ea7-99e0-c205b1e28ac9 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:23.469767 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c01cb5d-a4d7-430b-9c18-d9680c100e31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:23.470810 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01cb5d-a4d7-430b-9c18-d9680c100e31 service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:23.471103 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01cb5d-a4d7-430b-9c18-d9680c100e31 service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:23.471475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01cb5d-a4d7-430b-9c18-d9680c100e31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:23.473568 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-0c01cb5d-a4d7-430b-9c18-d9680c100e31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:23.549211 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c01cb5d-a4d7-430b-9c18-d9680c100e31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:23.550810 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c01cb5d-a4d7-430b-9c18-d9680c100e31 service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:23.561388 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 751/1516] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:23.562334 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c01cb5d-a4d7-430b-9c18-d9680c100e31 service cinder] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:23.563925 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-0c01cb5d-a4d7-430b-9c18-d9680c100e31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:23.572918 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866cba8f-182b-443b-a3f4-765bad445588 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:23.579412 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7aac3f-1995-4a06-96b4-b747a36c580c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:23.580454 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7aac3f-1995-4a06-96b4-b747a36c580c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:23.580880 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7aac3f-1995-4a06-96b4-b747a36c580c admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:23.581352 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7aac3f-1995-4a06-96b4-b747a36c580c admin admin] PATH_INFO: `/v3/projects/2578ef03784242c3843c6db2d6e87e82` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:23.582792 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7aac3f-1995-4a06-96b4-b747a36c580c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2578ef03784242c3843c6db2d6e87e82)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:23.583177 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c01cb5d-a4d7-430b-9c18-d9680c100e31 service cinder] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:23.585178 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7aac3f-1995-4a06-96b4-b747a36c580c admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:23.611439 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83235651-a769-44f7-8681-4ee352edad96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:23.612213 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83235651-a769-44f7-8681-4ee352edad96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:23.612582 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83235651-a769-44f7-8681-4ee352edad96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:23.612945 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83235651-a769-44f7-8681-4ee352edad96 admin admin] PATH_INFO: `/v3/projects/e72cb1596f56407cada971aaeb7fd599` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:23.614269 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83235651-a769-44f7-8681-4ee352edad96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e72cb1596f56407cada971aaeb7fd599)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:23.618086 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83235651-a769-44f7-8681-4ee352edad96 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:23.642433 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:23.656463 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-2f7aac3f-1995-4a06-96b4-b747a36c580c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2578ef03784242c3843c6db2d6e87e82', 'request_id': 'req-2f7aac3f-1995-4a06-96b4-b747a36c580c'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:23.671110 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-2f7aac3f-1995-4a06-96b4-b747a36c580c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2578ef03784242c3843c6db2d6e87e82', 'request_id': 'req-2f7aac3f-1995-4a06-96b4-b747a36c580c'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:23.680363 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-2f7aac3f-1995-4a06-96b4-b747a36c580c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2578ef03784242c3843c6db2d6e87e82', 'request_id': 'req-2f7aac3f-1995-4a06-96b4-b747a36c580c'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:23.692336 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-2f7aac3f-1995-4a06-96b4-b747a36c580c admin admin] The token cache is being invalidate because project 2578ef03784242c3843c6db2d6e87e82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:23.692669 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-2f7aac3f-1995-4a06-96b4-b747a36c580c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:23.693360 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-2f7aac3f-1995-4a06-96b4-b747a36c580c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:23.695153 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 766/1517] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:23 2022] DELETE /identity/v3/projects/2578ef03784242c3843c6db2d6e87e82 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:23.697549 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:23.699926 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa7a39-0751-4d8e-b954-d0d5813a059e None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:23.700217 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa7a39-0751-4d8e-b954-d0d5813a059e None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:23.700491 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa7a39-0751-4d8e-b954-d0d5813a059e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:23.708866 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-83235651-a769-44f7-8681-4ee352edad96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e72cb1596f56407cada971aaeb7fd599', 'request_id': 'req-83235651-a769-44f7-8681-4ee352edad96'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:23.720931 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-83235651-a769-44f7-8681-4ee352edad96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e72cb1596f56407cada971aaeb7fd599', 'request_id': 'req-83235651-a769-44f7-8681-4ee352edad96'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:23.723712 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-72aa7a39-0751-4d8e-b954-d0d5813a059e None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:23.725349 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-72aa7a39-0751-4d8e-b954-d0d5813a059e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:23.729374 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-83235651-a769-44f7-8681-4ee352edad96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e72cb1596f56407cada971aaeb7fd599', 'request_id': 'req-83235651-a769-44f7-8681-4ee352edad96'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:23.752765 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-83235651-a769-44f7-8681-4ee352edad96 admin admin] The token cache is being invalidate because project e72cb1596f56407cada971aaeb7fd599 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:23.753071 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-83235651-a769-44f7-8681-4ee352edad96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:23.754107 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-83235651-a769-44f7-8681-4ee352edad96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:23.756637 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 752/1518] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:23 2022] DELETE /identity/v3/projects/e72cb1596f56407cada971aaeb7fd599 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:23.762403 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83235651-a769-44f7-8681-4ee352edad96 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:23.764916 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-83235651-a769-44f7-8681-4ee352edad96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:23.820824 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 767/1519] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:23.827683 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260754d5-959e-4d60-92f6-804f9f52be35 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:23.828219 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260754d5-959e-4d60-92f6-804f9f52be35 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:23.828560 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260754d5-959e-4d60-92f6-804f9f52be35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:23.869074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-260754d5-959e-4d60-92f6-804f9f52be35 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:23.873640 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-260754d5-959e-4d60-92f6-804f9f52be35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:23.887045 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83235651-a769-44f7-8681-4ee352edad96 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:23.893136 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 768/1520] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:23.893342 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00912e16-7f54-4f82-8fee-4ac41bfe1b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:23.894138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00912e16-7f54-4f82-8fee-4ac41bfe1b56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:23.894413 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00912e16-7f54-4f82-8fee-4ac41bfe1b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:23.894757 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00912e16-7f54-4f82-8fee-4ac41bfe1b56 admin admin] PATH_INFO: `/v3/users/727f9b62f85747cc843cc5a90d5d0b01` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:23.895957 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00912e16-7f54-4f82-8fee-4ac41bfe1b56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=727f9b62f85747cc843cc5a90d5d0b01)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:23.898008 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00912e16-7f54-4f82-8fee-4ac41bfe1b56 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:23.898591 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260754d5-959e-4d60-92f6-804f9f52be35 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:23.908991 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260754d5-959e-4d60-92f6-804f9f52be35 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:23.916331 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decf19a7-c5f6-43b0-b75e-153be9769c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:23.917159 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decf19a7-c5f6-43b0-b75e-153be9769c40 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:23.917518 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decf19a7-c5f6-43b0-b75e-153be9769c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:23.917905 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decf19a7-c5f6-43b0-b75e-153be9769c40 admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:23.918717 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decf19a7-c5f6-43b0-b75e-153be9769c40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:23.919332 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decf19a7-c5f6-43b0-b75e-153be9769c40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:23.922680 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decf19a7-c5f6-43b0-b75e-153be9769c40 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:23.936860 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 769/1521] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:28:23.943124 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decf19a7-c5f6-43b0-b75e-153be9769c40 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:23.967405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decf19a7-c5f6-43b0-b75e-153be9769c40 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:23.973804 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5431b197-ada3-4925-840f-40817403864c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:23.974176 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431b197-ada3-4925-840f-40817403864c admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:23.974436 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431b197-ada3-4925-840f-40817403864c admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:23.974739 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431b197-ada3-4925-840f-40817403864c admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:23.975417 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5431b197-ada3-4925-840f-40817403864c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:23.977733 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5431b197-ada3-4925-840f-40817403864c admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:24.018942 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 770/1522] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:23 2022] POST /identity/v3/projects => generated 381 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:24.021958 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-00912e16-7f54-4f82-8fee-4ac41bfe1b56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '727f9b62f85747cc843cc5a90d5d0b01'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:24.022684 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-00912e16-7f54-4f82-8fee-4ac41bfe1b56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '727f9b62f85747cc843cc5a90d5d0b01'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:24.023839 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5431b197-ada3-4925-840f-40817403864c admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:24.026600 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-5431b197-ada3-4925-840f-40817403864c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:24.032865 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-00912e16-7f54-4f82-8fee-4ac41bfe1b56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '727f9b62f85747cc843cc5a90d5d0b01'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:24.050799 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-00912e16-7f54-4f82-8fee-4ac41bfe1b56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '727f9b62f85747cc843cc5a90d5d0b01'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:24.064124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-00912e16-7f54-4f82-8fee-4ac41bfe1b56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '727f9b62f85747cc843cc5a90d5d0b01'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:24.074697 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 753/1523] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:23 2022] DELETE /identity/v3/users/727f9b62f85747cc843cc5a90d5d0b01 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:24.124655 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5431b197-ada3-4925-840f-40817403864c admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:24.172126 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:24.205314 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb7cdd-6257-4821-aacb-5873f0f7949c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:24.206227 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb7cdd-6257-4821-aacb-5873f0f7949c admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:24.206351 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb7cdd-6257-4821-aacb-5873f0f7949c admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:24.206743 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb7cdd-6257-4821-aacb-5873f0f7949c admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:24.207385 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceb7cdd-6257-4821-aacb-5873f0f7949c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:24.209313 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceb7cdd-6257-4821-aacb-5873f0f7949c admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:24.243181 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 771/1524] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:24 2022] POST /identity/v3/users => generated 291 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:24.244575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00912e16-7f54-4f82-8fee-4ac41bfe1b56 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:24.257265 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00912e16-7f54-4f82-8fee-4ac41bfe1b56 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:24.269292 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e495cca5-411c-4c0a-ba30-012a1be06000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:24.269941 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495cca5-411c-4c0a-ba30-012a1be06000 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:24.270222 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495cca5-411c-4c0a-ba30-012a1be06000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:24.270537 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495cca5-411c-4c0a-ba30-012a1be06000 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:24.271174 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e495cca5-411c-4c0a-ba30-012a1be06000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:24.272831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e495cca5-411c-4c0a-ba30-012a1be06000 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:24.299647 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 754/1525] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:24.303425 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb7cdd-6257-4821-aacb-5873f0f7949c admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:24.321397 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb7cdd-6257-4821-aacb-5873f0f7949c admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:24.331374 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1132ef3d-ca67-4b50-a03f-ca5309db7333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:24.332112 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132ef3d-ca67-4b50-a03f-ca5309db7333 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:24.332451 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132ef3d-ca67-4b50-a03f-ca5309db7333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:24.333154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132ef3d-ca67-4b50-a03f-ca5309db7333 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:24.333542 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1132ef3d-ca67-4b50-a03f-ca5309db7333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:24.336943 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1132ef3d-ca67-4b50-a03f-ca5309db7333 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:24.353350 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 772/1526] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:24.357374 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e495cca5-411c-4c0a-ba30-012a1be06000 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:24.371697 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e495cca5-411c-4c0a-ba30-012a1be06000 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:24.381229 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd25325-7ff6-44b4-9b80-153ba43f4934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:24.381895 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd25325-7ff6-44b4-9b80-153ba43f4934 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:24.382241 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd25325-7ff6-44b4-9b80-153ba43f4934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:24.382712 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd25325-7ff6-44b4-9b80-153ba43f4934 admin admin] PATH_INFO: `/v3/projects/01da1608ea65414abf975e1f2d2adc74/users/9e62b0dc608541f48b02157c5219f40b/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:24.405751 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd25325-7ff6-44b4-9b80-153ba43f4934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01da1608ea65414abf975e1f2d2adc74, user_id=9e62b0dc608541f48b02157c5219f40b, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:24.408652 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd25325-7ff6-44b4-9b80-153ba43f4934 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:24.446972 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 755/1527] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:24 2022] PUT /identity/v3/projects/01da1608ea65414abf975e1f2d2adc74/users/9e62b0dc608541f48b02157c5219f40b/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:24.449419 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df323662-7aff-471d-a96f-0dff92b12183 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:24.449822 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df323662-7aff-471d-a96f-0dff92b12183 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:24.450164 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df323662-7aff-471d-a96f-0dff92b12183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:24.506588 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-df323662-7aff-471d-a96f-0dff92b12183 None None] MFA Rules not processed for user `9e62b0dc608541f48b02157c5219f40b`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:24.508113 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-df323662-7aff-471d-a96f-0dff92b12183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:24.528572 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd25325-7ff6-44b4-9b80-153ba43f4934 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:24.531272 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-cdd25325-7ff6-44b4-9b80-153ba43f4934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:24.551479 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:24.628838 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 773/1528] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:24.633429 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df323662-7aff-471d-a96f-0dff92b12183 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:24.635126 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd25325-7ff6-44b4-9b80-153ba43f4934 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:24.639696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3477a97-3ecc-4378-88da-3788637335b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:24.640338 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3477a97-3ecc-4378-88da-3788637335b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:24.640615 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3477a97-3ecc-4378-88da-3788637335b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:24.640885 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3477a97-3ecc-4378-88da-3788637335b0 admin admin] PATH_INFO: `/v3/projects/e87c2aa8fdba49b9bdc2b06b895c34ca` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:24.641999 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3477a97-3ecc-4378-88da-3788637335b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e87c2aa8fdba49b9bdc2b06b895c34ca)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:24.643394 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df323662-7aff-471d-a96f-0dff92b12183 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:24.643988 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3477a97-3ecc-4378-88da-3788637335b0 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:24.647886 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921f8fd-5862-4b98-9fca-08813ca7405e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:24.648597 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921f8fd-5862-4b98-9fca-08813ca7405e admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:24.648869 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921f8fd-5862-4b98-9fca-08813ca7405e admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:24.649307 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921f8fd-5862-4b98-9fca-08813ca7405e admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:24.649929 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8921f8fd-5862-4b98-9fca-08813ca7405e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:24.651833 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8921f8fd-5862-4b98-9fca-08813ca7405e admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:24.685980 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 774/1529] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:24 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:28:24.687161 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921f8fd-5862-4b98-9fca-08813ca7405e admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:24.701959 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a3477a97-3ecc-4378-88da-3788637335b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e87c2aa8fdba49b9bdc2b06b895c34ca', 'request_id': 'req-a3477a97-3ecc-4378-88da-3788637335b0'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:24.709005 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a3477a97-3ecc-4378-88da-3788637335b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e87c2aa8fdba49b9bdc2b06b895c34ca', 'request_id': 'req-a3477a97-3ecc-4378-88da-3788637335b0'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:24.715153 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a3477a97-3ecc-4378-88da-3788637335b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e87c2aa8fdba49b9bdc2b06b895c34ca', 'request_id': 'req-a3477a97-3ecc-4378-88da-3788637335b0'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:24.732370 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a3477a97-3ecc-4378-88da-3788637335b0 admin admin] The token cache is being invalidate because project e87c2aa8fdba49b9bdc2b06b895c34ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:24.732631 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a3477a97-3ecc-4378-88da-3788637335b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:24.733344 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a3477a97-3ecc-4378-88da-3788637335b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:24.734947 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 756/1530] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:24 2022] DELETE /identity/v3/projects/e87c2aa8fdba49b9bdc2b06b895c34ca => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:24.739724 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945132c-33f8-4459-b939-4c9909329f98 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:24.739999 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945132c-33f8-4459-b939-4c9909329f98 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:24.740264 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945132c-33f8-4459-b939-4c9909329f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:24.759788 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921f8fd-5862-4b98-9fca-08813ca7405e admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:24.760738 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-1945132c-33f8-4459-b939-4c9909329f98 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:24.761989 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-1945132c-33f8-4459-b939-4c9909329f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:24.786005 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:24.819047 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949e033-f492-4ae8-bbef-f932fbfd3f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:24.819659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949e033-f492-4ae8-bbef-f932fbfd3f65 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:24.819975 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949e033-f492-4ae8-bbef-f932fbfd3f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:24.820388 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949e033-f492-4ae8-bbef-f932fbfd3f65 admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:24.821051 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5949e033-f492-4ae8-bbef-f932fbfd3f65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:24.823137 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5949e033-f492-4ae8-bbef-f932fbfd3f65 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:24.826890 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 757/1531] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:24.834994 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ec718-1ac7-4a0f-b332-dc2bd044f8d0 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:24.835273 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ec718-1ac7-4a0f-b332-dc2bd044f8d0 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:24.835751 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ec718-1ac7-4a0f-b332-dc2bd044f8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:24.850492 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 775/1532] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:24 2022] POST /identity/v3/users => generated 298 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:24.851538 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949e033-f492-4ae8-bbef-f932fbfd3f65 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:24.853267 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-5949e033-f492-4ae8-bbef-f932fbfd3f65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:24.857433 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-9c0ec718-1ac7-4a0f-b332-dc2bd044f8d0 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:24.858587 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-9c0ec718-1ac7-4a0f-b332-dc2bd044f8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:24.865161 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949e033-f492-4ae8-bbef-f932fbfd3f65 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:24.870697 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 758/1533] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:24.871804 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ec718-1ac7-4a0f-b332-dc2bd044f8d0 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:24.873614 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91714d92-5e65-4486-9785-732f74f1b3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:24.874235 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91714d92-5e65-4486-9785-732f74f1b3bc admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:24.874517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91714d92-5e65-4486-9785-732f74f1b3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:24.874815 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91714d92-5e65-4486-9785-732f74f1b3bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:24.875463 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91714d92-5e65-4486-9785-732f74f1b3bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:24.877430 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91714d92-5e65-4486-9785-732f74f1b3bc admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:24.881527 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ec718-1ac7-4a0f-b332-dc2bd044f8d0 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:24.886324 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d836cb-846e-4653-bc72-3a2db53e3ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:24.887140 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d836cb-846e-4653-bc72-3a2db53e3ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:24.887410 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d836cb-846e-4653-bc72-3a2db53e3ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:24.887669 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d836cb-846e-4653-bc72-3a2db53e3ef6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:24.887829 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 776/1534] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:28:24.888295 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d836cb-846e-4653-bc72-3a2db53e3ef6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:24.888662 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d836cb-846e-4653-bc72-3a2db53e3ef6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:24.890282 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d836cb-846e-4653-bc72-3a2db53e3ef6 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:24.891966 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91714d92-5e65-4486-9785-732f74f1b3bc admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:24.902709 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91714d92-5e65-4486-9785-732f74f1b3bc admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:24.902837 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 759/1535] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:28:24.907118 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d836cb-846e-4653-bc72-3a2db53e3ef6 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:24.907968 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061467c-41a4-443d-8c3e-a225f7b6e858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:24.908623 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061467c-41a4-443d-8c3e-a225f7b6e858 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:24.908901 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061467c-41a4-443d-8c3e-a225f7b6e858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:24.909170 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061467c-41a4-443d-8c3e-a225f7b6e858 admin admin] PATH_INFO: `/v3/projects/a46b9d70dd1a4414a054770ce9369e40/users/03a5c2ddba4f43f4812e35b61fc64bfe/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:24.916813 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d836cb-846e-4653-bc72-3a2db53e3ef6 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:24.921687 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f5ae0-2875-4167-967e-508fe4876262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:24.922288 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f5ae0-2875-4167-967e-508fe4876262 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:24.922673 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f5ae0-2875-4167-967e-508fe4876262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:24.922955 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f5ae0-2875-4167-967e-508fe4876262 admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:24.923568 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f5ae0-2875-4167-967e-508fe4876262 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:24.925504 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f5ae0-2875-4167-967e-508fe4876262 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:24.925684 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061467c-41a4-443d-8c3e-a225f7b6e858 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a46b9d70dd1a4414a054770ce9369e40, user_id=03a5c2ddba4f43f4812e35b61fc64bfe, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:24.928422 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061467c-41a4-443d-8c3e-a225f7b6e858 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:24.943062 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 777/1536] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:24 2022] PUT /identity/v3/projects/a46b9d70dd1a4414a054770ce9369e40/users/03a5c2ddba4f43f4812e35b61fc64bfe/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:24.947244 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061467c-41a4-443d-8c3e-a225f7b6e858 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:24.957661 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 760/1537] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:24 2022] POST /identity/v3/projects => generated 391 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:28:24.958631 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f5ae0-2875-4167-967e-508fe4876262 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:24.963043 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:25.015499 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061467c-41a4-443d-8c3e-a225f7b6e858 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:25.028258 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f5ae0-2875-4167-967e-508fe4876262 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:25.033619 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a3ec4c-1e24-4101-a991-fb28840ffa09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:25.035944 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3ec4c-1e24-4101-a991-fb28840ffa09 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:25.036999 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3ec4c-1e24-4101-a991-fb28840ffa09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:25.036999 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3ec4c-1e24-4101-a991-fb28840ffa09 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:25.037200 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a3ec4c-1e24-4101-a991-fb28840ffa09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:25.039737 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a3ec4c-1e24-4101-a991-fb28840ffa09 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:25.077424 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 761/1538] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:24 2022] POST /identity/v3/users => generated 296 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:25.080457 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a3ec4c-1e24-4101-a991-fb28840ffa09 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:25.093813 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a3ec4c-1e24-4101-a991-fb28840ffa09 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:25.096270 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b470f-66c7-498a-a2f0-2552104dacf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:25.096896 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b470f-66c7-498a-a2f0-2552104dacf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:25.097312 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b470f-66c7-498a-a2f0-2552104dacf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:25.097652 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b470f-66c7-498a-a2f0-2552104dacf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:25.098243 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534b470f-66c7-498a-a2f0-2552104dacf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:25.099072 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aae3d3-db15-4fbd-ad1a-c03307cb9fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:25.099771 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aae3d3-db15-4fbd-ad1a-c03307cb9fa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:25.099954 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534b470f-66c7-498a-a2f0-2552104dacf1 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:25.100147 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aae3d3-db15-4fbd-ad1a-c03307cb9fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:25.100563 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aae3d3-db15-4fbd-ad1a-c03307cb9fa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:25.101488 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aae3d3-db15-4fbd-ad1a-c03307cb9fa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:25.103570 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aae3d3-db15-4fbd-ad1a-c03307cb9fa4 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:25.111426 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 778/1539] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:28:25.114915 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 762/1540] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:25.115540 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b470f-66c7-498a-a2f0-2552104dacf1 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:25.119056 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aae3d3-db15-4fbd-ad1a-c03307cb9fa4 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:25.126298 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b470f-66c7-498a-a2f0-2552104dacf1 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:25.129504 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aae3d3-db15-4fbd-ad1a-c03307cb9fa4 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:25.135844 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd93d095-daab-4a93-ba06-4d71961ed39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:25.136558 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd93d095-daab-4a93-ba06-4d71961ed39c admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:25.136880 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd93d095-daab-4a93-ba06-4d71961ed39c admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:25.137138 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3445f-4900-40b6-bf86-85abadcd70c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:25.137273 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd93d095-daab-4a93-ba06-4d71961ed39c admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:25.137785 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd93d095-daab-4a93-ba06-4d71961ed39c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:25.137907 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3445f-4900-40b6-bf86-85abadcd70c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:25.138079 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3445f-4900-40b6-bf86-85abadcd70c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:25.138350 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3445f-4900-40b6-bf86-85abadcd70c7 admin admin] PATH_INFO: `/v3/projects/a46b9d70dd1a4414a054770ce9369e40/users/03a5c2ddba4f43f4812e35b61fc64bfe/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:25.139795 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd93d095-daab-4a93-ba06-4d71961ed39c admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:25.140323 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f3445f-4900-40b6-bf86-85abadcd70c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a46b9d70dd1a4414a054770ce9369e40, user_id=03a5c2ddba4f43f4812e35b61fc64bfe, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:25.144820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f3445f-4900-40b6-bf86-85abadcd70c7 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:25.151298 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 763/1541] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:28:25.166224 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd93d095-daab-4a93-ba06-4d71961ed39c admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:25.174238 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 779/1542] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:25 2022] PUT /identity/v3/projects/a46b9d70dd1a4414a054770ce9369e40/users/03a5c2ddba4f43f4812e35b61fc64bfe/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:25.176320 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd93d095-daab-4a93-ba06-4d71961ed39c admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:25.185642 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23250b-a08d-4ae5-95d8-ec8dc68da424 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:25.185928 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23250b-a08d-4ae5-95d8-ec8dc68da424 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:25.186202 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23250b-a08d-4ae5-95d8-ec8dc68da424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:25.212149 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:25.215012 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-0f23250b-a08d-4ae5-95d8-ec8dc68da424 None None] MFA Rules not processed for user `03a5c2ddba4f43f4812e35b61fc64bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:25.216953 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-0f23250b-a08d-4ae5-95d8-ec8dc68da424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:25.243857 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c42ab0-601b-47cb-a804-64acd059d401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:25.244524 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c42ab0-601b-47cb-a804-64acd059d401 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:25.244798 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c42ab0-601b-47cb-a804-64acd059d401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:25.245066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c42ab0-601b-47cb-a804-64acd059d401 admin admin] PATH_INFO: `/v3/projects/50e5f6a3fc2c41d0aafc104ae021ecd3/users/8da8f1644cf94242986c2fdf8a6e04a3/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:25.261271 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c42ab0-601b-47cb-a804-64acd059d401 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50e5f6a3fc2c41d0aafc104ae021ecd3, user_id=8da8f1644cf94242986c2fdf8a6e04a3, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:25.263992 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c42ab0-601b-47cb-a804-64acd059d401 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:25.289178 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 764/1543] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:25 2022] PUT /identity/v3/projects/50e5f6a3fc2c41d0aafc104ae021ecd3/users/8da8f1644cf94242986c2fdf8a6e04a3/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:25.299215 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 780/1544] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:25 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:25.300364 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b674539-dfd9-4e51-be73-ccccc0a9dd27 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:25.300364 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b674539-dfd9-4e51-be73-ccccc0a9dd27 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:25.301537 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b674539-dfd9-4e51-be73-ccccc0a9dd27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:25.314462 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23250b-a08d-4ae5-95d8-ec8dc68da424 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:25.316144 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-0f23250b-a08d-4ae5-95d8-ec8dc68da424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:25.349360 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-8b674539-dfd9-4e51-be73-ccccc0a9dd27 None None] MFA Rules not processed for user `8da8f1644cf94242986c2fdf8a6e04a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:25.350680 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-8b674539-dfd9-4e51-be73-ccccc0a9dd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:25.404966 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:25.425769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23250b-a08d-4ae5-95d8-ec8dc68da424 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:25.464174 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 765/1545] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:25 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:25.468883 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b674539-dfd9-4e51-be73-ccccc0a9dd27 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:25.498166 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8feb0c-a009-4216-8406-9eaeedb45e94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:25.498989 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8feb0c-a009-4216-8406-9eaeedb45e94 service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:25.499279 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8feb0c-a009-4216-8406-9eaeedb45e94 service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:25.499532 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8feb0c-a009-4216-8406-9eaeedb45e94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:25.550155 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b674539-dfd9-4e51-be73-ccccc0a9dd27 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:25.557385 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef5d28-2428-45e7-978a-021e0653e31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:25.557934 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef5d28-2428-45e7-978a-021e0653e31e admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:25.560851 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef5d28-2428-45e7-978a-021e0653e31e admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:25.561139 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef5d28-2428-45e7-978a-021e0653e31e admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:25.561826 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ef5d28-2428-45e7-978a-021e0653e31e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:25.563818 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ef5d28-2428-45e7-978a-021e0653e31e admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:25.603886 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 766/1546] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:25 2022] POST /identity/v3/projects => generated 391 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:28:25.615481 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef5d28-2428-45e7-978a-021e0653e31e admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:25.624070 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8feb0c-a009-4216-8406-9eaeedb45e94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:25.625536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8feb0c-a009-4216-8406-9eaeedb45e94 service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:25.643294 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 781/1547] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:25 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:25.774715 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef5d28-2428-45e7-978a-021e0653e31e admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:25.782071 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dedc5b-a309-4ea2-a4c5-ddab64664847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:25.782928 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dedc5b-a309-4ea2-a4c5-ddab64664847 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:25.783191 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dedc5b-a309-4ea2-a4c5-ddab64664847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:25.783510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dedc5b-a309-4ea2-a4c5-ddab64664847 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:25.784165 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dedc5b-a309-4ea2-a4c5-ddab64664847 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:25.786171 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dedc5b-a309-4ea2-a4c5-ddab64664847 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:25.809126 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8feb0c-a009-4216-8406-9eaeedb45e94 service cinder] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:25.821708 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 767/1548] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:25 2022] POST /identity/v3/users => generated 302 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:25.826322 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dedc5b-a309-4ea2-a4c5-ddab64664847 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:25.835919 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:25.840445 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dedc5b-a309-4ea2-a4c5-ddab64664847 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:25.842578 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8feb0c-a009-4216-8406-9eaeedb45e94 service cinder] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:25.847494 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe769d-9f77-481d-918f-eed518ccc076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:25.850114 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe769d-9f77-481d-918f-eed518ccc076 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:25.850489 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe769d-9f77-481d-918f-eed518ccc076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:25.850913 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe769d-9f77-481d-918f-eed518ccc076 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:25.852459 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbe769d-9f77-481d-918f-eed518ccc076 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:25.854541 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbe769d-9f77-481d-918f-eed518ccc076 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:25.874563 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 768/1549] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:25.876514 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe769d-9f77-481d-918f-eed518ccc076 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:25.889079 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe769d-9f77-481d-918f-eed518ccc076 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:25.894446 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801572ba-9af2-4c58-b134-57e9c6c88335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:25.895131 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801572ba-9af2-4c58-b134-57e9c6c88335 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:25.895430 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801572ba-9af2-4c58-b134-57e9c6c88335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:25.896475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801572ba-9af2-4c58-b134-57e9c6c88335 admin admin] PATH_INFO: `/v3/projects/a158e2aceedf42de83c3704903e16a0b/users/3a826603ad0248729cbcd62a05a591fd/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:25.918094 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801572ba-9af2-4c58-b134-57e9c6c88335 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a158e2aceedf42de83c3704903e16a0b, user_id=3a826603ad0248729cbcd62a05a591fd, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:25.923103 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801572ba-9af2-4c58-b134-57e9c6c88335 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:25.942679 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 769/1550] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:25 2022] PUT /identity/v3/projects/a158e2aceedf42de83c3704903e16a0b/users/3a826603ad0248729cbcd62a05a591fd/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:25.947997 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801572ba-9af2-4c58-b134-57e9c6c88335 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:25.963419 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86c7a1-a0a1-4528-8b47-d275805c4537 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:25.964442 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86c7a1-a0a1-4528-8b47-d275805c4537 service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:25.965110 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86c7a1-a0a1-4528-8b47-d275805c4537 service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:25.965669 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86c7a1-a0a1-4528-8b47-d275805c4537 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:25.969042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-3d86c7a1-a0a1-4528-8b47-d275805c4537 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:25.987142 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:26.044232 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86c7a1-a0a1-4528-8b47-d275805c4537 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:26.045768 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86c7a1-a0a1-4528-8b47-d275805c4537 service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:26.056184 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 782/1551] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:28:26.064662 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801572ba-9af2-4c58-b134-57e9c6c88335 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:26.069276 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16596a77-0871-49fe-94de-b2da7cafca2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:26.069901 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16596a77-0871-49fe-94de-b2da7cafca2f admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:26.070199 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16596a77-0871-49fe-94de-b2da7cafca2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:26.070578 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16596a77-0871-49fe-94de-b2da7cafca2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:26.071125 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16596a77-0871-49fe-94de-b2da7cafca2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:26.072708 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16596a77-0871-49fe-94de-b2da7cafca2f admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:26.084204 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 770/1552] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:26.089011 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86c7a1-a0a1-4528-8b47-d275805c4537 service cinder] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:26.105601 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86c7a1-a0a1-4528-8b47-d275805c4537 service cinder] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:26.113043 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757bbf5-5a9e-4243-bc4c-b4ced4a31ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:26.113729 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757bbf5-5a9e-4243-bc4c-b4ced4a31ea2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:26.114010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757bbf5-5a9e-4243-bc4c-b4ced4a31ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:26.114278 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757bbf5-5a9e-4243-bc4c-b4ced4a31ea2 admin admin] PATH_INFO: `/v3/projects/a158e2aceedf42de83c3704903e16a0b/users/3a826603ad0248729cbcd62a05a591fd/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:26.117877 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757bbf5-5a9e-4243-bc4c-b4ced4a31ea2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a158e2aceedf42de83c3704903e16a0b, user_id=3a826603ad0248729cbcd62a05a591fd, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:26.121805 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757bbf5-5a9e-4243-bc4c-b4ced4a31ea2 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:26.138711 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 783/1553] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:26 2022] PUT /identity/v3/projects/a158e2aceedf42de83c3704903e16a0b/users/3a826603ad0248729cbcd62a05a591fd/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:26.150835 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f980ddf-6d99-4044-8b96-653b888ec8ad None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:26.151288 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f980ddf-6d99-4044-8b96-653b888ec8ad None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:26.151692 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f980ddf-6d99-4044-8b96-653b888ec8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:26.226096 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-1f980ddf-6d99-4044-8b96-653b888ec8ad None None] MFA Rules not processed for user `3a826603ad0248729cbcd62a05a591fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:26.228435 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-1f980ddf-6d99-4044-8b96-653b888ec8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:26.424700 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 771/1554] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:26 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:26.532774 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757bbf5-5a9e-4243-bc4c-b4ced4a31ea2 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:26.536083 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-c757bbf5-5a9e-4243-bc4c-b4ced4a31ea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:26.633135 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757bbf5-5a9e-4243-bc4c-b4ced4a31ea2 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:26.644773 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a40e4-95dd-4a5f-8f59-9bab33fcaa01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:26.645599 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a40e4-95dd-4a5f-8f59-9bab33fcaa01 service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:26.645987 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a40e4-95dd-4a5f-8f59-9bab33fcaa01 service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:26.646393 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a40e4-95dd-4a5f-8f59-9bab33fcaa01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:26.659336 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a40e4-95dd-4a5f-8f59-9bab33fcaa01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:26.660997 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a40e4-95dd-4a5f-8f59-9bab33fcaa01 service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:26.686562 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 784/1555] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:26 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:26.831645 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f980ddf-6d99-4044-8b96-653b888ec8ad None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:26.852667 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f980ddf-6d99-4044-8b96-653b888ec8ad None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:26.858036 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a9e3f-ff11-41fc-aba9-870801948a9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:26.858956 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a9e3f-ff11-41fc-aba9-870801948a9a service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:26.859242 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a9e3f-ff11-41fc-aba9-870801948a9a service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:26.859755 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a9e3f-ff11-41fc-aba9-870801948a9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:26.956023 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26a9e3f-ff11-41fc-aba9-870801948a9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:26.957850 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26a9e3f-ff11-41fc-aba9-870801948a9a service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:26.969312 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 772/1556] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:26 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:27.388325 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a40e4-95dd-4a5f-8f59-9bab33fcaa01 service cinder] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:27.406287 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a40e4-95dd-4a5f-8f59-9bab33fcaa01 service cinder] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:27.412869 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc8c24-ea20-4126-a0e1-a4fa16c850c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:27.413815 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc8c24-ea20-4126-a0e1-a4fa16c850c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:27.414301 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc8c24-ea20-4126-a0e1-a4fa16c850c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:27.414744 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc8c24-ea20-4126-a0e1-a4fa16c850c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:27.417123 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-d0fc8c24-ea20-4126-a0e1-a4fa16c850c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:27.527938 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:27.553815 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fc8c24-ea20-4126-a0e1-a4fa16c850c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:27.555360 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fc8c24-ea20-4126-a0e1-a4fa16c850c5 service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:27.578253 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 785/1557] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:28.783121 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a9e3f-ff11-41fc-aba9-870801948a9a service cinder] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:28.795283 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a9e3f-ff11-41fc-aba9-870801948a9a service cinder] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:28.803603 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d79c7f-d22b-4ec8-a983-38fbd6a87eea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:28.804387 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d79c7f-d22b-4ec8-a983-38fbd6a87eea service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:28.804907 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d79c7f-d22b-4ec8-a983-38fbd6a87eea service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:28.805479 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d79c7f-d22b-4ec8-a983-38fbd6a87eea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:28.810944 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-59d79c7f-d22b-4ec8-a983-38fbd6a87eea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:28.885508 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:28.914563 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d79c7f-d22b-4ec8-a983-38fbd6a87eea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:28.915990 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d79c7f-d22b-4ec8-a983-38fbd6a87eea service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:28.927090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 773/1558] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:28 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:32.625716 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376e1a7-f178-4feb-81de-d211aea530c8 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:32.626304 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376e1a7-f178-4feb-81de-d211aea530c8 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:32.626304 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376e1a7-f178-4feb-81de-d211aea530c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:32.649435 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-4376e1a7-f178-4feb-81de-d211aea530c8 None None] MFA Rules not processed for user `65d51ec74e1e4c25bcc4365eb858bac0`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:32.650832 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-4376e1a7-f178-4feb-81de-d211aea530c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:32.713104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 786/1559] 10.222.0.40 () {58 vars in 1015 bytes} [Wed Nov 2 07:28:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:33.059257 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d79c7f-d22b-4ec8-a983-38fbd6a87eea service cinder] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:33.061559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-59d79c7f-d22b-4ec8-a983-38fbd6a87eea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:33.062819 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376e1a7-f178-4feb-81de-d211aea530c8 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:33.065659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-4376e1a7-f178-4feb-81de-d211aea530c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:33.100732 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376e1a7-f178-4feb-81de-d211aea530c8 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:33.107418 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d79c7f-d22b-4ec8-a983-38fbd6a87eea service cinder] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:33.132969 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b729ce-ac4e-42d6-9d98-16e8feaea410 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:33.134071 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b729ce-ac4e-42d6-9d98-16e8feaea410 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:33.135960 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc5e3f1-09b7-4c19-9eed-9d1aae51deaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:33.136595 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5e3f1-09b7-4c19-9eed-9d1aae51deaf service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:33.136884 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5e3f1-09b7-4c19-9eed-9d1aae51deaf service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:33.137145 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5e3f1-09b7-4c19-9eed-9d1aae51deaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:33.137558 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b729ce-ac4e-42d6-9d98-16e8feaea410 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:33.137825 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b729ce-ac4e-42d6-9d98-16e8feaea410 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:33.147135 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc5e3f1-09b7-4c19-9eed-9d1aae51deaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:33.148636 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc5e3f1-09b7-4c19-9eed-9d1aae51deaf service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:33.154860 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b729ce-ac4e-42d6-9d98-16e8feaea410 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:33.156936 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b729ce-ac4e-42d6-9d98-16e8feaea410 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:33.158704 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 774/1560] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:33.160203 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc5e3f1-09b7-4c19-9eed-9d1aae51deaf service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:33.194482 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 787/1561] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:33.257419 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc5e3f1-09b7-4c19-9eed-9d1aae51deaf service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:33.266506 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fa90ac-71de-48b0-952b-f1876b7f2ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:33.267557 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa90ac-71de-48b0-952b-f1876b7f2ed4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:33.267935 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa90ac-71de-48b0-952b-f1876b7f2ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:33.268364 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa90ac-71de-48b0-952b-f1876b7f2ed4 admin admin] PATH_INFO: `/v3/users/9e62b0dc608541f48b02157c5219f40b` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:33.269838 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fa90ac-71de-48b0-952b-f1876b7f2ed4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e62b0dc608541f48b02157c5219f40b)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:33.272331 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fa90ac-71de-48b0-952b-f1876b7f2ed4 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:33.371498 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-89fa90ac-71de-48b0-952b-f1876b7f2ed4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e62b0dc608541f48b02157c5219f40b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:33.373922 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-89fa90ac-71de-48b0-952b-f1876b7f2ed4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e62b0dc608541f48b02157c5219f40b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:33.393732 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-89fa90ac-71de-48b0-952b-f1876b7f2ed4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e62b0dc608541f48b02157c5219f40b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:33.434963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-89fa90ac-71de-48b0-952b-f1876b7f2ed4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e62b0dc608541f48b02157c5219f40b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:33.437434 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b729ce-ac4e-42d6-9d98-16e8feaea410 service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:33.449659 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-89fa90ac-71de-48b0-952b-f1876b7f2ed4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e62b0dc608541f48b02157c5219f40b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:33.454524 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b729ce-ac4e-42d6-9d98-16e8feaea410 service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:33.460421 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 775/1562] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:33 2022] DELETE /identity/v3/users/9e62b0dc608541f48b02157c5219f40b => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:33.462961 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bec6d2-1b36-43f7-b9e1-e5847fad59b1 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:33.463425 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bec6d2-1b36-43f7-b9e1-e5847fad59b1 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:33.463736 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bec6d2-1b36-43f7-b9e1-e5847fad59b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:33.467228 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3749c-cf62-48ad-91db-7058ed4640b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:33.468227 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3749c-cf62-48ad-91db-7058ed4640b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:33.468943 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3749c-cf62-48ad-91db-7058ed4640b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:33.469419 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3749c-cf62-48ad-91db-7058ed4640b4 admin admin] PATH_INFO: `/v3/users/8da8f1644cf94242986c2fdf8a6e04a3` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:33.471134 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da3749c-cf62-48ad-91db-7058ed4640b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8da8f1644cf94242986c2fdf8a6e04a3)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:33.474843 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da3749c-cf62-48ad-91db-7058ed4640b4 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:33.499648 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-77bec6d2-1b36-43f7-b9e1-e5847fad59b1 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:33.501187 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-77bec6d2-1b36-43f7-b9e1-e5847fad59b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:33.558856 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:33.613178 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5da3749c-cf62-48ad-91db-7058ed4640b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8da8f1644cf94242986c2fdf8a6e04a3'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:33.631638 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5da3749c-cf62-48ad-91db-7058ed4640b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8da8f1644cf94242986c2fdf8a6e04a3'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:33.632710 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5da3749c-cf62-48ad-91db-7058ed4640b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8da8f1644cf94242986c2fdf8a6e04a3'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:33.633682 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 776/1563] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:33.655320 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bec6d2-1b36-43f7-b9e1-e5847fad59b1 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:33.657357 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-77bec6d2-1b36-43f7-b9e1-e5847fad59b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:33.663734 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5da3749c-cf62-48ad-91db-7058ed4640b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8da8f1644cf94242986c2fdf8a6e04a3'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:33.689226 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5da3749c-cf62-48ad-91db-7058ed4640b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8da8f1644cf94242986c2fdf8a6e04a3'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:33.703902 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 788/1564] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:33 2022] DELETE /identity/v3/users/8da8f1644cf94242986c2fdf8a6e04a3 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:33.708669 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fec179-d240-4b0c-be0d-8c275164c01e None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:33.708962 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fec179-d240-4b0c-be0d-8c275164c01e None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:33.709237 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fec179-d240-4b0c-be0d-8c275164c01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:33.775279 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-b1fec179-d240-4b0c-be0d-8c275164c01e None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:33.782521 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-b1fec179-d240-4b0c-be0d-8c275164c01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:33.836113 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bec6d2-1b36-43f7-b9e1-e5847fad59b1 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:33.896838 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:33.929380 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 789/1565] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:33.948679 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ab74da-5bb5-4a78-a3cc-176623986d50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:33.949650 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab74da-5bb5-4a78-a3cc-176623986d50 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:33.949961 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab74da-5bb5-4a78-a3cc-176623986d50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:33.951186 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab74da-5bb5-4a78-a3cc-176623986d50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:33.953416 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-99ab74da-5bb5-4a78-a3cc-176623986d50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:33.976131 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ab74da-5bb5-4a78-a3cc-176623986d50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:33.978337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ab74da-5bb5-4a78-a3cc-176623986d50 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:33.991783 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 777/1566] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:28:34.002831 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fec179-d240-4b0c-be0d-8c275164c01e None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:34.004421 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-b1fec179-d240-4b0c-be0d-8c275164c01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:34.016597 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fec179-d240-4b0c-be0d-8c275164c01e None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:34.020718 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b2e7e-17ec-4d33-bd8b-c245152471bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:34.021340 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b2e7e-17ec-4d33-bd8b-c245152471bd service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:34.021612 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b2e7e-17ec-4d33-bd8b-c245152471bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:34.021873 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b2e7e-17ec-4d33-bd8b-c245152471bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:34.032086 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2b2e7e-17ec-4d33-bd8b-c245152471bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:34.033800 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2b2e7e-17ec-4d33-bd8b-c245152471bd service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:34.047413 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 790/1567] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:34.755240 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdecae8-db2f-4954-9d31-92aa8811d6d4 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:34.755854 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdecae8-db2f-4954-9d31-92aa8811d6d4 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:34.756043 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdecae8-db2f-4954-9d31-92aa8811d6d4 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:34.758256 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 778/1568] 10.222.0.30 () {54 vars in 921 bytes} [Wed Nov 2 07:28:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:28:34.769109 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaca081-3181-4632-ab15-109debcd2a94 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:34.771285 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaca081-3181-4632-ab15-109debcd2a94 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:34.771728 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaca081-3181-4632-ab15-109debcd2a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:34.797683 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-bfaca081-3181-4632-ab15-109debcd2a94 None None] MFA Rules not processed for user `e6495385dccc4fa48c05b1feb957846c`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:34.799659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-bfaca081-3181-4632-ab15-109debcd2a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:34.829125 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdecae8-db2f-4954-9d31-92aa8811d6d4 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:34.831238 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-cfdecae8-db2f-4954-9d31-92aa8811d6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:34.860775 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdecae8-db2f-4954-9d31-92aa8811d6d4 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:34.865739 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ac236-eb7e-4373-96a1-20e401f8f8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:34.887886 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ac236-eb7e-4373-96a1-20e401f8f8ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:34.888210 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ac236-eb7e-4373-96a1-20e401f8f8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:34.888487 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ac236-eb7e-4373-96a1-20e401f8f8ee admin admin] PATH_INFO: `/v3/projects/01da1608ea65414abf975e1f2d2adc74` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:34.891288 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ac236-eb7e-4373-96a1-20e401f8f8ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=01da1608ea65414abf975e1f2d2adc74)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:34.899997 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ac236-eb7e-4373-96a1-20e401f8f8ee admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:34.921019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 791/1569] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Nov 2 07:28:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:34.934453 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaca081-3181-4632-ab15-109debcd2a94 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:34.935992 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-bfaca081-3181-4632-ab15-109debcd2a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:34.963466 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaca081-3181-4632-ab15-109debcd2a94 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:34.976462 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e0d17e-e1f2-4442-846c-7a68bc2a6f72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:34.977464 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0d17e-e1f2-4442-846c-7a68bc2a6f72 service nova] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:34.978245 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0d17e-e1f2-4442-846c-7a68bc2a6f72 service nova] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:34.979525 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0d17e-e1f2-4442-846c-7a68bc2a6f72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:34.994890 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e0d17e-e1f2-4442-846c-7a68bc2a6f72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:34.996920 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e0d17e-e1f2-4442-846c-7a68bc2a6f72 service nova] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:35.009798 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-984ac236-eb7e-4373-96a1-20e401f8f8ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01da1608ea65414abf975e1f2d2adc74', 'request_id': 'req-984ac236-eb7e-4373-96a1-20e401f8f8ee'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:35.011632 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 792/1570] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:35.012849 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e0d17e-e1f2-4442-846c-7a68bc2a6f72 service nova] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:35.015615 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-f6e0d17e-e1f2-4442-846c-7a68bc2a6f72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:35.023757 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-984ac236-eb7e-4373-96a1-20e401f8f8ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01da1608ea65414abf975e1f2d2adc74', 'request_id': 'req-984ac236-eb7e-4373-96a1-20e401f8f8ee'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:35.034154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e0d17e-e1f2-4442-846c-7a68bc2a6f72 service nova] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:35.043588 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-984ac236-eb7e-4373-96a1-20e401f8f8ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01da1608ea65414abf975e1f2d2adc74', 'request_id': 'req-984ac236-eb7e-4373-96a1-20e401f8f8ee'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:35.063388 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-984ac236-eb7e-4373-96a1-20e401f8f8ee admin admin] The token cache is being invalidate because project 01da1608ea65414abf975e1f2d2adc74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:35.063787 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-984ac236-eb7e-4373-96a1-20e401f8f8ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:35.064405 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-984ac236-eb7e-4373-96a1-20e401f8f8ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:35.069272 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 779/1571] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:34 2022] DELETE /identity/v3/projects/01da1608ea65414abf975e1f2d2adc74 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:35.070814 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ac236-eb7e-4373-96a1-20e401f8f8ee admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:35.072247 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-984ac236-eb7e-4373-96a1-20e401f8f8ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:35.109367 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:35.135715 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf513f16-4ae0-4768-bc2e-bb679106e830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:35.136646 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf513f16-4ae0-4768-bc2e-bb679106e830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:35.136750 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf513f16-4ae0-4768-bc2e-bb679106e830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:35.137179 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf513f16-4ae0-4768-bc2e-bb679106e830 admin admin] PATH_INFO: `/v3/projects/50e5f6a3fc2c41d0aafc104ae021ecd3` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:35.138575 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf513f16-4ae0-4768-bc2e-bb679106e830 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50e5f6a3fc2c41d0aafc104ae021ecd3)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:35.140650 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf513f16-4ae0-4768-bc2e-bb679106e830 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:35.217787 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ac236-eb7e-4373-96a1-20e401f8f8ee admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:35.223731 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-cf513f16-4ae0-4768-bc2e-bb679106e830 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50e5f6a3fc2c41d0aafc104ae021ecd3', 'request_id': 'req-cf513f16-4ae0-4768-bc2e-bb679106e830'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:35.254747 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-cf513f16-4ae0-4768-bc2e-bb679106e830 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50e5f6a3fc2c41d0aafc104ae021ecd3', 'request_id': 'req-cf513f16-4ae0-4768-bc2e-bb679106e830'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:35.265203 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:35.268783 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-cf513f16-4ae0-4768-bc2e-bb679106e830 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50e5f6a3fc2c41d0aafc104ae021ecd3', 'request_id': 'req-cf513f16-4ae0-4768-bc2e-bb679106e830'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:35.295728 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-cf513f16-4ae0-4768-bc2e-bb679106e830 admin admin] The token cache is being invalidate because project 50e5f6a3fc2c41d0aafc104ae021ecd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:35.296019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-cf513f16-4ae0-4768-bc2e-bb679106e830 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:35.306768 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-cf513f16-4ae0-4768-bc2e-bb679106e830 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:35.306768 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 793/1572] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:35 2022] DELETE /identity/v3/projects/50e5f6a3fc2c41d0aafc104ae021ecd3 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:35.311955 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf513f16-4ae0-4768-bc2e-bb679106e830 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:35.315175 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c832080-6b84-4b88-89cd-27b0695a9383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:35.315822 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-cf513f16-4ae0-4768-bc2e-bb679106e830 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:35.316607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c832080-6b84-4b88-89cd-27b0695a9383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:35.317211 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c832080-6b84-4b88-89cd-27b0695a9383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:35.317629 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c832080-6b84-4b88-89cd-27b0695a9383 admin admin] PATH_INFO: `/v3/users/03a5c2ddba4f43f4812e35b61fc64bfe` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:35.319918 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c832080-6b84-4b88-89cd-27b0695a9383 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03a5c2ddba4f43f4812e35b61fc64bfe)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:35.323270 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c832080-6b84-4b88-89cd-27b0695a9383 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:35.344915 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf513f16-4ae0-4768-bc2e-bb679106e830 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:35.351218 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abeee22-d1e9-4eea-9be7-b22aee4e6dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:35.352560 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abeee22-d1e9-4eea-9be7-b22aee4e6dfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:35.353025 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abeee22-d1e9-4eea-9be7-b22aee4e6dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:35.353985 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abeee22-d1e9-4eea-9be7-b22aee4e6dfa admin admin] PATH_INFO: `/v3/users/3a826603ad0248729cbcd62a05a591fd` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:35.355223 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abeee22-d1e9-4eea-9be7-b22aee4e6dfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a826603ad0248729cbcd62a05a591fd)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:35.359696 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abeee22-d1e9-4eea-9be7-b22aee4e6dfa admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:35.426824 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-5c832080-6b84-4b88-89cd-27b0695a9383 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03a5c2ddba4f43f4812e35b61fc64bfe'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:35.428084 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-5c832080-6b84-4b88-89cd-27b0695a9383 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03a5c2ddba4f43f4812e35b61fc64bfe'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:35.439803 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0abeee22-d1e9-4eea-9be7-b22aee4e6dfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a826603ad0248729cbcd62a05a591fd'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:35.446062 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-5c832080-6b84-4b88-89cd-27b0695a9383 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03a5c2ddba4f43f4812e35b61fc64bfe'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:35.450231 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0abeee22-d1e9-4eea-9be7-b22aee4e6dfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a826603ad0248729cbcd62a05a591fd'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:35.451728 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0abeee22-d1e9-4eea-9be7-b22aee4e6dfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a826603ad0248729cbcd62a05a591fd'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:35.465083 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-5c832080-6b84-4b88-89cd-27b0695a9383 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03a5c2ddba4f43f4812e35b61fc64bfe'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:35.476792 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-5c832080-6b84-4b88-89cd-27b0695a9383 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03a5c2ddba4f43f4812e35b61fc64bfe'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:35.477049 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0abeee22-d1e9-4eea-9be7-b22aee4e6dfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a826603ad0248729cbcd62a05a591fd'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:35.486954 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0abeee22-d1e9-4eea-9be7-b22aee4e6dfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a826603ad0248729cbcd62a05a591fd'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:35.493656 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 780/1573] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:35 2022] DELETE /identity/v3/users/03a5c2ddba4f43f4812e35b61fc64bfe => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:35.494256 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 794/1574] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:35 2022] DELETE /identity/v3/users/3a826603ad0248729cbcd62a05a591fd => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:35.970641 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a3d425-2c0f-4ab0-9ccd-ad7cb3e27d26 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:35.972340 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a3d425-2c0f-4ab0-9ccd-ad7cb3e27d26 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:35.973113 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a3d425-2c0f-4ab0-9ccd-ad7cb3e27d26 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:35.975320 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 781/1575] 10.222.0.30 () {54 vars in 921 bytes} [Wed Nov 2 07:28:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:28:35.988140 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2aeb81-adaa-46b3-b06c-1e4332f3c85f None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:35.988785 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2aeb81-adaa-46b3-b06c-1e4332f3c85f None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:35.989763 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2aeb81-adaa-46b3-b06c-1e4332f3c85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.042771 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-fc2aeb81-adaa-46b3-b06c-1e4332f3c85f None None] MFA Rules not processed for user `e6495385dccc4fa48c05b1feb957846c`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:36.044019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-fc2aeb81-adaa-46b3-b06c-1e4332f3c85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:36.123674 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 795/1576] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Nov 2 07:28:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:36.136320 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a3d425-2c0f-4ab0-9ccd-ad7cb3e27d26 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:36.138060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-a3a3d425-2c0f-4ab0-9ccd-ad7cb3e27d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:36.155744 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a3d425-2c0f-4ab0-9ccd-ad7cb3e27d26 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:36.161244 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27828793-d96c-42a2-8bf5-0a096cb0ca6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:36.161861 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27828793-d96c-42a2-8bf5-0a096cb0ca6f service nova] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.162135 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27828793-d96c-42a2-8bf5-0a096cb0ca6f service nova] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.162399 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27828793-d96c-42a2-8bf5-0a096cb0ca6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.171772 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27828793-d96c-42a2-8bf5-0a096cb0ca6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:36.173469 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27828793-d96c-42a2-8bf5-0a096cb0ca6f service nova] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:36.186209 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 782/1577] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:36.198382 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2aeb81-adaa-46b3-b06c-1e4332f3c85f None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:36.200333 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-fc2aeb81-adaa-46b3-b06c-1e4332f3c85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:36.206213 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27828793-d96c-42a2-8bf5-0a096cb0ca6f service nova] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:36.208631 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-27828793-d96c-42a2-8bf5-0a096cb0ca6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:36.282503 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:36.288985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:36.289456 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2aeb81-adaa-46b3-b06c-1e4332f3c85f None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:36.295699 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7cf9e6-c7e4-493f-b619-b2cb021e924f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:36.296544 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cf9e6-c7e4-493f-b619-b2cb021e924f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.297087 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cf9e6-c7e4-493f-b619-b2cb021e924f admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.297362 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cf9e6-c7e4-493f-b619-b2cb021e924f admin admin] PATH_INFO: `/v3/projects/a158e2aceedf42de83c3704903e16a0b` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.298631 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7cf9e6-c7e4-493f-b619-b2cb021e924f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a158e2aceedf42de83c3704903e16a0b)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:36.300655 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7cf9e6-c7e4-493f-b619-b2cb021e924f admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:36.315275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27828793-d96c-42a2-8bf5-0a096cb0ca6f service nova] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:36.321017 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c390e9-be46-48fd-a991-5d7623d9bddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:36.321674 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c390e9-be46-48fd-a991-5d7623d9bddc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.321948 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c390e9-be46-48fd-a991-5d7623d9bddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.322365 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c390e9-be46-48fd-a991-5d7623d9bddc admin admin] PATH_INFO: `/v3/projects/a46b9d70dd1a4414a054770ce9369e40` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.323657 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c390e9-be46-48fd-a991-5d7623d9bddc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a46b9d70dd1a4414a054770ce9369e40)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:36.325694 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c390e9-be46-48fd-a991-5d7623d9bddc admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:36.350457 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-8d7cf9e6-c7e4-493f-b619-b2cb021e924f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a158e2aceedf42de83c3704903e16a0b', 'request_id': 'req-8d7cf9e6-c7e4-493f-b619-b2cb021e924f'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:36.360161 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-8d7cf9e6-c7e4-493f-b619-b2cb021e924f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a158e2aceedf42de83c3704903e16a0b', 'request_id': 'req-8d7cf9e6-c7e4-493f-b619-b2cb021e924f'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:36.366200 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-8d7cf9e6-c7e4-493f-b619-b2cb021e924f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a158e2aceedf42de83c3704903e16a0b', 'request_id': 'req-8d7cf9e6-c7e4-493f-b619-b2cb021e924f'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:36.377767 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-e0c390e9-be46-48fd-a991-5d7623d9bddc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a46b9d70dd1a4414a054770ce9369e40', 'request_id': 'req-e0c390e9-be46-48fd-a991-5d7623d9bddc'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:36.378425 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-8d7cf9e6-c7e4-493f-b619-b2cb021e924f admin admin] The token cache is being invalidate because project a158e2aceedf42de83c3704903e16a0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:36.378793 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-8d7cf9e6-c7e4-493f-b619-b2cb021e924f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:36.379432 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-8d7cf9e6-c7e4-493f-b619-b2cb021e924f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:36.381022 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 796/1578] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:36 2022] DELETE /identity/v3/projects/a158e2aceedf42de83c3704903e16a0b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:36.384554 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-e0c390e9-be46-48fd-a991-5d7623d9bddc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a46b9d70dd1a4414a054770ce9369e40', 'request_id': 'req-e0c390e9-be46-48fd-a991-5d7623d9bddc'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:36.386235 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6300a10-db46-424f-a777-abe1497872a4 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.386518 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6300a10-db46-424f-a777-abe1497872a4 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.386815 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6300a10-db46-424f-a777-abe1497872a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.390459 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-e0c390e9-be46-48fd-a991-5d7623d9bddc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a46b9d70dd1a4414a054770ce9369e40', 'request_id': 'req-e0c390e9-be46-48fd-a991-5d7623d9bddc'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:36.404779 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-e0c390e9-be46-48fd-a991-5d7623d9bddc admin admin] The token cache is being invalidate because project a46b9d70dd1a4414a054770ce9369e40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:36.405180 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-e0c390e9-be46-48fd-a991-5d7623d9bddc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:36.405832 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-e0c390e9-be46-48fd-a991-5d7623d9bddc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:36.407637 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 783/1579] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:36 2022] DELETE /identity/v3/projects/a46b9d70dd1a4414a054770ce9369e40 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:36.408033 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-b6300a10-db46-424f-a777-abe1497872a4 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:36.409312 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-b6300a10-db46-424f-a777-abe1497872a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:36.412569 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32df2dd-00b1-46c3-813d-b772de0d0e07 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.412844 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32df2dd-00b1-46c3-813d-b772de0d0e07 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.413106 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32df2dd-00b1-46c3-813d-b772de0d0e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.433146 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-d32df2dd-00b1-46c3-813d-b772de0d0e07 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:36.434335 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-d32df2dd-00b1-46c3-813d-b772de0d0e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:36.476286 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 797/1580] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:36.484052 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99212c-c742-4092-b51b-45a9d7fdc0ae None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.484334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99212c-c742-4092-b51b-45a9d7fdc0ae None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.484602 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99212c-c742-4092-b51b-45a9d7fdc0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.495416 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 784/1581] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:36.503378 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1c433-bf22-4608-b2b3-6c98ee7fefb6 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.503651 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1c433-bf22-4608-b2b3-6c98ee7fefb6 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.503917 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1c433-bf22-4608-b2b3-6c98ee7fefb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.504986 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-dd99212c-c742-4092-b51b-45a9d7fdc0ae None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:36.506214 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-dd99212c-c742-4092-b51b-45a9d7fdc0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:36.514098 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 798/1582] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:36.518599 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99212c-c742-4092-b51b-45a9d7fdc0ae None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:36.523268 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-1cb1c433-bf22-4608-b2b3-6c98ee7fefb6 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:36.524622 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-1cb1c433-bf22-4608-b2b3-6c98ee7fefb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:36.529418 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99212c-c742-4092-b51b-45a9d7fdc0ae None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:36.533860 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 785/1583] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:36.535132 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1724dc7-2f0c-40b9-9070-636e710cdf23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:36.535706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1724dc7-2f0c-40b9-9070-636e710cdf23 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.535985 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1724dc7-2f0c-40b9-9070-636e710cdf23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.536388 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1724dc7-2f0c-40b9-9070-636e710cdf23 admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.537162 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1724dc7-2f0c-40b9-9070-636e710cdf23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:36.537470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1724dc7-2f0c-40b9-9070-636e710cdf23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:36.538559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1c433-bf22-4608-b2b3-6c98ee7fefb6 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:36.539111 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1724dc7-2f0c-40b9-9070-636e710cdf23 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:36.548084 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1c433-bf22-4608-b2b3-6c98ee7fefb6 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:36.552705 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df6a0cc-3155-4b44-9f5f-0ead4f2a7fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:36.553310 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6a0cc-3155-4b44-9f5f-0ead4f2a7fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.553581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6a0cc-3155-4b44-9f5f-0ead4f2a7fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.553846 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6a0cc-3155-4b44-9f5f-0ead4f2a7fc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.554479 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df6a0cc-3155-4b44-9f5f-0ead4f2a7fc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:36.554820 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df6a0cc-3155-4b44-9f5f-0ead4f2a7fc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:36.555015 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 799/1584] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:28:36.556701 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df6a0cc-3155-4b44-9f5f-0ead4f2a7fc1 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:36.559059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1724dc7-2f0c-40b9-9070-636e710cdf23 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:36.570024 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1724dc7-2f0c-40b9-9070-636e710cdf23 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:36.570838 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 786/1585] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:28:36.574422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df6a0cc-3155-4b44-9f5f-0ead4f2a7fc1 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:36.578042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49152952-961c-4d7e-ba8b-858fbc3897ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:36.579540 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49152952-961c-4d7e-ba8b-858fbc3897ec admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.580617 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49152952-961c-4d7e-ba8b-858fbc3897ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.581173 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49152952-961c-4d7e-ba8b-858fbc3897ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.582023 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49152952-961c-4d7e-ba8b-858fbc3897ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:36.584430 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49152952-961c-4d7e-ba8b-858fbc3897ec admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:36.590346 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df6a0cc-3155-4b44-9f5f-0ead4f2a7fc1 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:36.597156 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bbba3-ea2e-48fd-bdb2-6a9fd36bfd42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:36.597944 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bbba3-ea2e-48fd-bdb2-6a9fd36bfd42 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.598266 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bbba3-ea2e-48fd-bdb2-6a9fd36bfd42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.598608 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bbba3-ea2e-48fd-bdb2-6a9fd36bfd42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.600005 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799bbba3-ea2e-48fd-bdb2-6a9fd36bfd42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:36.603682 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799bbba3-ea2e-48fd-bdb2-6a9fd36bfd42 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:36.632967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 800/1586] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:36 2022] POST /identity/v3/projects => generated 371 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:28:36.639154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49152952-961c-4d7e-ba8b-858fbc3897ec admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:36.651661 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 787/1587] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:36 2022] POST /identity/v3/projects => generated 387 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:36.653546 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bbba3-ea2e-48fd-bdb2-6a9fd36bfd42 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:36.702540 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:36.711302 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:36.724327 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49152952-961c-4d7e-ba8b-858fbc3897ec admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:36.751728 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bbba3-ea2e-48fd-bdb2-6a9fd36bfd42 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:36.756315 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab672d0f-845b-4d0e-8b2f-4ba73118387e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:36.756999 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab672d0f-845b-4d0e-8b2f-4ba73118387e admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.757341 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab672d0f-845b-4d0e-8b2f-4ba73118387e admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.757674 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab672d0f-845b-4d0e-8b2f-4ba73118387e admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.758366 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab672d0f-845b-4d0e-8b2f-4ba73118387e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:36.760839 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab672d0f-845b-4d0e-8b2f-4ba73118387e admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:36.789031 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 788/1588] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:36 2022] POST /identity/v3/users => generated 294 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:36.792893 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f0507-8cf5-479c-adbc-e49fd03f7850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:36.793573 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f0507-8cf5-479c-adbc-e49fd03f7850 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.793876 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f0507-8cf5-479c-adbc-e49fd03f7850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.793981 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab672d0f-845b-4d0e-8b2f-4ba73118387e admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:36.796073 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f0507-8cf5-479c-adbc-e49fd03f7850 admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.796073 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4f0507-8cf5-479c-adbc-e49fd03f7850 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:36.796985 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4f0507-8cf5-479c-adbc-e49fd03f7850 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:36.804696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab672d0f-845b-4d0e-8b2f-4ba73118387e admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:36.812573 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936a297-2110-4891-a164-73f850dd8b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:36.813186 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936a297-2110-4891-a164-73f850dd8b5c admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.813459 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936a297-2110-4891-a164-73f850dd8b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.813719 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936a297-2110-4891-a164-73f850dd8b5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.814362 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7936a297-2110-4891-a164-73f850dd8b5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:36.816024 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7936a297-2110-4891-a164-73f850dd8b5c admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:36.830480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 789/1589] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:36.830704 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 801/1590] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:36 2022] POST /identity/v3/users => generated 286 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:36.832236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936a297-2110-4891-a164-73f850dd8b5c admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:36.832485 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f0507-8cf5-479c-adbc-e49fd03f7850 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:36.846761 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936a297-2110-4891-a164-73f850dd8b5c admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:36.847740 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f0507-8cf5-479c-adbc-e49fd03f7850 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:36.852310 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c9c59-efc8-4571-937d-cf2639eabb76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:36.853004 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c9c59-efc8-4571-937d-cf2639eabb76 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.853383 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c9c59-efc8-4571-937d-cf2639eabb76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.853985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c9c59-efc8-4571-937d-cf2639eabb76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.854709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115c9c59-efc8-4571-937d-cf2639eabb76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:36.856469 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115c9c59-efc8-4571-937d-cf2639eabb76 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:36.857224 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2049086-88d4-4826-aa11-7a148289d182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:36.857842 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2049086-88d4-4826-aa11-7a148289d182 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.858128 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2049086-88d4-4826-aa11-7a148289d182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.858376 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2049086-88d4-4826-aa11-7a148289d182 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.859319 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2049086-88d4-4826-aa11-7a148289d182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:36.860627 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2049086-88d4-4826-aa11-7a148289d182 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:36.870941 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 790/1591] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:36.871749 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 802/1592] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:36.872943 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c9c59-efc8-4571-937d-cf2639eabb76 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:36.875582 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2049086-88d4-4826-aa11-7a148289d182 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:36.883538 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c9c59-efc8-4571-937d-cf2639eabb76 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:36.888735 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2049086-88d4-4826-aa11-7a148289d182 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:36.892026 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62582bf-7cc5-4ed3-936c-c7034e4ee245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:36.892891 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62582bf-7cc5-4ed3-936c-c7034e4ee245 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.893170 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62582bf-7cc5-4ed3-936c-c7034e4ee245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.893437 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62582bf-7cc5-4ed3-936c-c7034e4ee245 admin admin] PATH_INFO: `/v3/projects/9a0409b1c7a94617b59b3a4e89e6b84d/users/f8bfbb5f8bbf42c88701ca06c2441eb6/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.893787 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306aeb05-3389-4222-833d-04c42f1728aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:36.894604 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306aeb05-3389-4222-833d-04c42f1728aa admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.894973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306aeb05-3389-4222-833d-04c42f1728aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.895302 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306aeb05-3389-4222-833d-04c42f1728aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.896005 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306aeb05-3389-4222-833d-04c42f1728aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:36.897670 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306aeb05-3389-4222-833d-04c42f1728aa admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:36.910707 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 803/1593] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:28:36.913101 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62582bf-7cc5-4ed3-936c-c7034e4ee245 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a0409b1c7a94617b59b3a4e89e6b84d, user_id=f8bfbb5f8bbf42c88701ca06c2441eb6, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:36.914261 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306aeb05-3389-4222-833d-04c42f1728aa admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:36.915880 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62582bf-7cc5-4ed3-936c-c7034e4ee245 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:36.925904 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306aeb05-3389-4222-833d-04c42f1728aa admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:36.931238 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fdbef2-bffa-479c-847d-7381bfb86e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:36.932051 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fdbef2-bffa-479c-847d-7381bfb86e4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.932522 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fdbef2-bffa-479c-847d-7381bfb86e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.932814 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fdbef2-bffa-479c-847d-7381bfb86e4c admin admin] PATH_INFO: `/v3/projects/20c55579c8b44160ba38f1d5337c64c2/users/d1ab3a393c3e4e1eb0bd6865f7080548/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.938706 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 791/1594] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:36 2022] PUT /identity/v3/projects/9a0409b1c7a94617b59b3a4e89e6b84d/users/f8bfbb5f8bbf42c88701ca06c2441eb6/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:36.945912 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80400388-8c53-45b0-86eb-1bfdf76e3d81 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.946430 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80400388-8c53-45b0-86eb-1bfdf76e3d81 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.946746 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80400388-8c53-45b0-86eb-1bfdf76e3d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.948406 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fdbef2-bffa-479c-847d-7381bfb86e4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=20c55579c8b44160ba38f1d5337c64c2, user_id=d1ab3a393c3e4e1eb0bd6865f7080548, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:36.951163 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fdbef2-bffa-479c-847d-7381bfb86e4c admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:36.975939 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 804/1595] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:36 2022] PUT /identity/v3/projects/20c55579c8b44160ba38f1d5337c64c2/users/d1ab3a393c3e4e1eb0bd6865f7080548/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:36.981206 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d235b47-4e8a-4042-affd-6028423c415f None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:36.981486 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d235b47-4e8a-4042-affd-6028423c415f None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:36.981749 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d235b47-4e8a-4042-affd-6028423c415f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:36.984125 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-80400388-8c53-45b0-86eb-1bfdf76e3d81 None None] MFA Rules not processed for user `f8bfbb5f8bbf42c88701ca06c2441eb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:36.985406 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-80400388-8c53-45b0-86eb-1bfdf76e3d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:37.000179 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-3d235b47-4e8a-4042-affd-6028423c415f None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:37.001696 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-3d235b47-4e8a-4042-affd-6028423c415f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:37.051844 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 792/1596] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:37.055986 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80400388-8c53-45b0-86eb-1bfdf76e3d81 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:37.067098 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 805/1597] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:37.073995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d235b47-4e8a-4042-affd-6028423c415f None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:37.075613 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-3d235b47-4e8a-4042-affd-6028423c415f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:37.117054 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:37.121811 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80400388-8c53-45b0-86eb-1bfdf76e3d81 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:37.126759 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f0f46-88cd-409d-80ce-4fc3f16715d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:37.127364 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f0f46-88cd-409d-80ce-4fc3f16715d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:37.127663 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f0f46-88cd-409d-80ce-4fc3f16715d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:37.127926 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f0f46-88cd-409d-80ce-4fc3f16715d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:37.128545 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44f0f46-88cd-409d-80ce-4fc3f16715d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:37.130437 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44f0f46-88cd-409d-80ce-4fc3f16715d2 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:37.135267 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:37.135587 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d235b47-4e8a-4042-affd-6028423c415f None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:37.139762 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c29af-8090-4395-80b4-1a1b129f2699 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:37.140373 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c29af-8090-4395-80b4-1a1b129f2699 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:37.140945 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c29af-8090-4395-80b4-1a1b129f2699 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:37.141215 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c29af-8090-4395-80b4-1a1b129f2699 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:37.150763 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52c29af-8090-4395-80b4-1a1b129f2699 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:37.152678 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52c29af-8090-4395-80b4-1a1b129f2699 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:37.166636 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 793/1598] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:37 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:28:37.166809 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 806/1599] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:28:37.167787 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f0f46-88cd-409d-80ce-4fc3f16715d2 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:37.235560 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f0f46-88cd-409d-80ce-4fc3f16715d2 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:37.241318 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5066384-13ed-4de2-ad91-6c85dfa730c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:37.241984 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5066384-13ed-4de2-ad91-6c85dfa730c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:37.242177 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5066384-13ed-4de2-ad91-6c85dfa730c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:37.242501 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5066384-13ed-4de2-ad91-6c85dfa730c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:37.243101 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5066384-13ed-4de2-ad91-6c85dfa730c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:37.245552 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5066384-13ed-4de2-ad91-6c85dfa730c9 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:37.272199 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 794/1600] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:37 2022] POST /identity/v3/users => generated 300 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:37.276270 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c29af-8090-4395-80b4-1a1b129f2699 service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:37.286283 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c29af-8090-4395-80b4-1a1b129f2699 service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:37.291141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d70c2-e4c4-48ac-907d-357d55aad64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:37.291756 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d70c2-e4c4-48ac-907d-357d55aad64e admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:37.292031 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d70c2-e4c4-48ac-907d-357d55aad64e admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:37.292414 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d70c2-e4c4-48ac-907d-357d55aad64e admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:37.292935 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6d70c2-e4c4-48ac-907d-357d55aad64e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:37.294559 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6d70c2-e4c4-48ac-907d-357d55aad64e admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:37.305377 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 807/1601] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:37.309979 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5066384-13ed-4de2-ad91-6c85dfa730c9 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:37.321180 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5066384-13ed-4de2-ad91-6c85dfa730c9 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:37.326493 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297add9-25c4-408d-98a1-5622e47f6391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:37.327190 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297add9-25c4-408d-98a1-5622e47f6391 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:37.327463 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297add9-25c4-408d-98a1-5622e47f6391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:37.327730 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297add9-25c4-408d-98a1-5622e47f6391 admin admin] PATH_INFO: `/v3/projects/d5cccf8559be41d688dbb5eabdf4a78c/users/cdcd5135201c4174b320fa07c1604f48/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:37.345653 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3297add9-25c4-408d-98a1-5622e47f6391 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5cccf8559be41d688dbb5eabdf4a78c, user_id=cdcd5135201c4174b320fa07c1604f48, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:37.348357 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3297add9-25c4-408d-98a1-5622e47f6391 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:37.364399 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 795/1602] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:37 2022] PUT /identity/v3/projects/d5cccf8559be41d688dbb5eabdf4a78c/users/cdcd5135201c4174b320fa07c1604f48/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:37.368425 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d70c2-e4c4-48ac-907d-357d55aad64e admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:37.378933 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d70c2-e4c4-48ac-907d-357d55aad64e admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:37.438880 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0860b79-ed18-4333-bed2-5af11f50f455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:37.439499 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0860b79-ed18-4333-bed2-5af11f50f455 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:37.439770 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0860b79-ed18-4333-bed2-5af11f50f455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:37.440160 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0860b79-ed18-4333-bed2-5af11f50f455 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:37.440804 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0860b79-ed18-4333-bed2-5af11f50f455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:37.442429 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0860b79-ed18-4333-bed2-5af11f50f455 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:37.453883 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 808/1603] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:37.459685 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297add9-25c4-408d-98a1-5622e47f6391 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:37.470447 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297add9-25c4-408d-98a1-5622e47f6391 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:37.478301 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72ceee-11a3-47e5-9021-60f1f4cc1b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:37.481275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72ceee-11a3-47e5-9021-60f1f4cc1b8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:37.481275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72ceee-11a3-47e5-9021-60f1f4cc1b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:37.481578 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72ceee-11a3-47e5-9021-60f1f4cc1b8b admin admin] PATH_INFO: `/v3/projects/d5cccf8559be41d688dbb5eabdf4a78c/users/cdcd5135201c4174b320fa07c1604f48/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:37.488755 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba72ceee-11a3-47e5-9021-60f1f4cc1b8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5cccf8559be41d688dbb5eabdf4a78c, user_id=cdcd5135201c4174b320fa07c1604f48, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:37.491549 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba72ceee-11a3-47e5-9021-60f1f4cc1b8b admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:37.512202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 796/1604] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:37 2022] PUT /identity/v3/projects/d5cccf8559be41d688dbb5eabdf4a78c/users/cdcd5135201c4174b320fa07c1604f48/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:37.526796 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4bb3f7-dbb1-40b3-a790-1bc6a8070439 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:37.527198 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4bb3f7-dbb1-40b3-a790-1bc6a8070439 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:37.527601 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4bb3f7-dbb1-40b3-a790-1bc6a8070439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:37.562692 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-ed4bb3f7-dbb1-40b3-a790-1bc6a8070439 None None] MFA Rules not processed for user `cdcd5135201c4174b320fa07c1604f48`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:37.563918 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-ed4bb3f7-dbb1-40b3-a790-1bc6a8070439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:37.679301 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 809/1605] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:37 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:37.721050 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72ceee-11a3-47e5-9021-60f1f4cc1b8b admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:37.723797 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-ba72ceee-11a3-47e5-9021-60f1f4cc1b8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:37.801964 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72ceee-11a3-47e5-9021-60f1f4cc1b8b admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:37.807156 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34395c5b-78fe-419e-b7ae-efb5634ec1b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:37.807900 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34395c5b-78fe-419e-b7ae-efb5634ec1b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:37.808274 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34395c5b-78fe-419e-b7ae-efb5634ec1b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:37.808818 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34395c5b-78fe-419e-b7ae-efb5634ec1b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:37.822013 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34395c5b-78fe-419e-b7ae-efb5634ec1b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:37.824017 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34395c5b-78fe-419e-b7ae-efb5634ec1b7 service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:37.836695 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 797/1606] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:37 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:38.021264 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4bb3f7-dbb1-40b3-a790-1bc6a8070439 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:38.083596 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:38.102341 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4bb3f7-dbb1-40b3-a790-1bc6a8070439 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:38.109778 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584493a4-788a-4cde-80f8-25562eb16a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:38.110440 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584493a4-788a-4cde-80f8-25562eb16a57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:38.110753 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584493a4-788a-4cde-80f8-25562eb16a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:38.111015 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584493a4-788a-4cde-80f8-25562eb16a57 admin admin] PATH_INFO: `/v3/users/f8bfbb5f8bbf42c88701ca06c2441eb6` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:38.112182 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584493a4-788a-4cde-80f8-25562eb16a57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8bfbb5f8bbf42c88701ca06c2441eb6)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:38.114132 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584493a4-788a-4cde-80f8-25562eb16a57 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:38.199326 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-584493a4-788a-4cde-80f8-25562eb16a57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8bfbb5f8bbf42c88701ca06c2441eb6'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:38.207794 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-584493a4-788a-4cde-80f8-25562eb16a57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8bfbb5f8bbf42c88701ca06c2441eb6'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:38.208744 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-584493a4-788a-4cde-80f8-25562eb16a57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8bfbb5f8bbf42c88701ca06c2441eb6'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:38.225095 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-584493a4-788a-4cde-80f8-25562eb16a57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8bfbb5f8bbf42c88701ca06c2441eb6'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:38.248016 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-584493a4-788a-4cde-80f8-25562eb16a57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8bfbb5f8bbf42c88701ca06c2441eb6'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:38.263793 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 810/1607] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:38 2022] DELETE /identity/v3/users/f8bfbb5f8bbf42c88701ca06c2441eb6 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:38.265091 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5fc1a9-c4d7-4621-978e-27e84f03708f None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:38.265871 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5fc1a9-c4d7-4621-978e-27e84f03708f None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:38.266276 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5fc1a9-c4d7-4621-978e-27e84f03708f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:38.292100 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-8a5fc1a9-c4d7-4621-978e-27e84f03708f None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:38.294708 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-8a5fc1a9-c4d7-4621-978e-27e84f03708f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:38.352614 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:38.414134 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 798/1608] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:38.427640 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584493a4-788a-4cde-80f8-25562eb16a57 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:38.429111 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-584493a4-788a-4cde-80f8-25562eb16a57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:38.515733 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584493a4-788a-4cde-80f8-25562eb16a57 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:38.518981 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2bbbf-4283-4710-aa43-1621b8622023 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:38.521004 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2bbbf-4283-4710-aa43-1621b8622023 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:38.521256 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2bbbf-4283-4710-aa43-1621b8622023 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:38.522761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2bbbf-4283-4710-aa43-1621b8622023 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:38.537681 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a2bbbf-4283-4710-aa43-1621b8622023 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:38.539296 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a2bbbf-4283-4710-aa43-1621b8622023 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:38.553153 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 811/1609] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:38.657767 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d3729-4269-4afe-98d6-28d69d24e075 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:38.658049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d3729-4269-4afe-98d6-28d69d24e075 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:38.658314 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d3729-4269-4afe-98d6-28d69d24e075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:38.705739 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-f59d3729-4269-4afe-98d6-28d69d24e075 None None] MFA Rules not processed for user `d1ab3a393c3e4e1eb0bd6865f7080548`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:38.707120 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-f59d3729-4269-4afe-98d6-28d69d24e075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:38.905961 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 799/1610] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:38.910842 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2bbbf-4283-4710-aa43-1621b8622023 service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:38.913168 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-b9a2bbbf-4283-4710-aa43-1621b8622023 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:38.952507 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2bbbf-4283-4710-aa43-1621b8622023 service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:38.957195 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da61d01-d660-45d3-9406-55aa30007429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:38.957822 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da61d01-d660-45d3-9406-55aa30007429 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:38.958073 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da61d01-d660-45d3-9406-55aa30007429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:38.958346 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da61d01-d660-45d3-9406-55aa30007429 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:38.959019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da61d01-d660-45d3-9406-55aa30007429 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:38.960934 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da61d01-d660-45d3-9406-55aa30007429 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:39.040283 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 812/1611] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:38 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:39.041942 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d3729-4269-4afe-98d6-28d69d24e075 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:39.054879 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d3729-4269-4afe-98d6-28d69d24e075 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:39.073918 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da61d01-d660-45d3-9406-55aa30007429 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:39.076408 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-6da61d01-d660-45d3-9406-55aa30007429 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:39.159980 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536c3a1-fdec-4afe-a0d7-b38b904ae7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:39.160918 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536c3a1-fdec-4afe-a0d7-b38b904ae7f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:39.161267 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536c3a1-fdec-4afe-a0d7-b38b904ae7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:39.161610 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536c3a1-fdec-4afe-a0d7-b38b904ae7f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:39.169319 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c536c3a1-fdec-4afe-a0d7-b38b904ae7f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:39.172437 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c536c3a1-fdec-4afe-a0d7-b38b904ae7f7 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:39.195749 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:39.230121 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da61d01-d660-45d3-9406-55aa30007429 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:39.239508 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f129baf-ad69-47cb-8613-44413ea4e904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:39.240510 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f129baf-ad69-47cb-8613-44413ea4e904 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:39.240852 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 800/1612] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:39 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:28:39.241325 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f129baf-ad69-47cb-8613-44413ea4e904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:39.242940 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f129baf-ad69-47cb-8613-44413ea4e904 admin admin] PATH_INFO: `/v3/users/957c2507c03846208b79b8c262384d1d` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:39.245627 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536c3a1-fdec-4afe-a0d7-b38b904ae7f7 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:39.245944 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f129baf-ad69-47cb-8613-44413ea4e904 admin admin] RBAC: Authorizing `identity:delete_user(user_id=957c2507c03846208b79b8c262384d1d)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:39.249194 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f129baf-ad69-47cb-8613-44413ea4e904 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:39.259555 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536c3a1-fdec-4afe-a0d7-b38b904ae7f7 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:39.266556 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f73da1-0290-4568-b3d7-6328e3202c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:39.267247 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f73da1-0290-4568-b3d7-6328e3202c4d admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:39.267649 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f73da1-0290-4568-b3d7-6328e3202c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:39.268592 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f73da1-0290-4568-b3d7-6328e3202c4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:39.269395 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f73da1-0290-4568-b3d7-6328e3202c4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:39.271069 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f73da1-0290-4568-b3d7-6328e3202c4d admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:39.289392 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 801/1613] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:28:39.290498 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f73da1-0290-4568-b3d7-6328e3202c4d admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:39.303188 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f73da1-0290-4568-b3d7-6328e3202c4d admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:39.308472 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932a6d9-30b6-42d0-973e-b62304d50374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:39.310123 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932a6d9-30b6-42d0-973e-b62304d50374 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:39.310756 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932a6d9-30b6-42d0-973e-b62304d50374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:39.311209 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932a6d9-30b6-42d0-973e-b62304d50374 admin admin] PATH_INFO: `/v3/projects/86f2720035c54fefb0c1d4bb47661361/users/12e2a78ac61d4749ab8e20647e009c85/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:39.334503 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-7f129baf-ad69-47cb-8613-44413ea4e904 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '957c2507c03846208b79b8c262384d1d'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:39.348113 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-7f129baf-ad69-47cb-8613-44413ea4e904 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '957c2507c03846208b79b8c262384d1d'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:39.349775 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-7f129baf-ad69-47cb-8613-44413ea4e904 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '957c2507c03846208b79b8c262384d1d'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:39.352656 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2932a6d9-30b6-42d0-973e-b62304d50374 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86f2720035c54fefb0c1d4bb47661361, user_id=12e2a78ac61d4749ab8e20647e009c85, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:39.356079 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2932a6d9-30b6-42d0-973e-b62304d50374 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:39.376601 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-7f129baf-ad69-47cb-8613-44413ea4e904 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '957c2507c03846208b79b8c262384d1d'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:39.390580 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 802/1614] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:39 2022] PUT /identity/v3/projects/86f2720035c54fefb0c1d4bb47661361/users/12e2a78ac61d4749ab8e20647e009c85/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:39.393935 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-7f129baf-ad69-47cb-8613-44413ea4e904 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '957c2507c03846208b79b8c262384d1d'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:39.396715 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932a6d9-30b6-42d0-973e-b62304d50374 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:39.402396 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-2932a6d9-30b6-42d0-973e-b62304d50374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:39.412818 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 813/1615] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:39 2022] DELETE /identity/v3/users/957c2507c03846208b79b8c262384d1d => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:39.418358 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c5e4d-12d3-4f64-a37a-88b0277d1357 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:39.419218 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c5e4d-12d3-4f64-a37a-88b0277d1357 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:39.420127 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c5e4d-12d3-4f64-a37a-88b0277d1357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:39.462895 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-3c9c5e4d-12d3-4f64-a37a-88b0277d1357 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:39.475601 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-3c9c5e4d-12d3-4f64-a37a-88b0277d1357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:39.483347 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:39.568864 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932a6d9-30b6-42d0-973e-b62304d50374 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:39.636725 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 814/1616] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:39.638963 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c5e4d-12d3-4f64-a37a-88b0277d1357 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:39.644421 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-3c9c5e4d-12d3-4f64-a37a-88b0277d1357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:39.676334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c5e4d-12d3-4f64-a37a-88b0277d1357 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:39.689564 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bdf16e-69ab-4587-937f-fb74616124d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:39.690193 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bdf16e-69ab-4587-937f-fb74616124d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:39.690473 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bdf16e-69ab-4587-937f-fb74616124d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:39.690770 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bdf16e-69ab-4587-937f-fb74616124d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:39.691422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bdf16e-69ab-4587-937f-fb74616124d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:39.693392 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867eb12-4e3f-4399-956f-e2917b93f568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:39.694091 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867eb12-4e3f-4399-956f-e2917b93f568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:39.694376 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867eb12-4e3f-4399-956f-e2917b93f568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:39.695208 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867eb12-4e3f-4399-956f-e2917b93f568 admin admin] PATH_INFO: `/v3/projects/9a0409b1c7a94617b59b3a4e89e6b84d` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:39.696777 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0867eb12-4e3f-4399-956f-e2917b93f568 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a0409b1c7a94617b59b3a4e89e6b84d)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:39.698989 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0867eb12-4e3f-4399-956f-e2917b93f568 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:39.702002 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bdf16e-69ab-4587-937f-fb74616124d2 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:39.733871 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 803/1617] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:39.748156 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bdf16e-69ab-4587-937f-fb74616124d2 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:39.750283 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-72bdf16e-69ab-4587-937f-fb74616124d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:39.798635 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0867eb12-4e3f-4399-956f-e2917b93f568 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a0409b1c7a94617b59b3a4e89e6b84d', 'request_id': 'req-0867eb12-4e3f-4399-956f-e2917b93f568'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:39.816117 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0867eb12-4e3f-4399-956f-e2917b93f568 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a0409b1c7a94617b59b3a4e89e6b84d', 'request_id': 'req-0867eb12-4e3f-4399-956f-e2917b93f568'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:39.828146 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:39.830643 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0867eb12-4e3f-4399-956f-e2917b93f568 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a0409b1c7a94617b59b3a4e89e6b84d', 'request_id': 'req-0867eb12-4e3f-4399-956f-e2917b93f568'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:39.849165 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0867eb12-4e3f-4399-956f-e2917b93f568 admin admin] The token cache is being invalidate because project 9a0409b1c7a94617b59b3a4e89e6b84d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:39.850252 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0867eb12-4e3f-4399-956f-e2917b93f568 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:39.851241 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0867eb12-4e3f-4399-956f-e2917b93f568 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:39.856295 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 815/1618] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:39 2022] DELETE /identity/v3/projects/9a0409b1c7a94617b59b3a4e89e6b84d => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:39.856295 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867eb12-4e3f-4399-956f-e2917b93f568 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:39.858152 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-0867eb12-4e3f-4399-956f-e2917b93f568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:39.892453 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bdf16e-69ab-4587-937f-fb74616124d2 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:39.956104 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747a09e-1ba4-4513-8459-f40cfc3f4d77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:39.957052 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747a09e-1ba4-4513-8459-f40cfc3f4d77 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:39.957230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747a09e-1ba4-4513-8459-f40cfc3f4d77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:39.957506 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747a09e-1ba4-4513-8459-f40cfc3f4d77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:39.959729 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-e747a09e-1ba4-4513-8459-f40cfc3f4d77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:39.960777 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867eb12-4e3f-4399-956f-e2917b93f568 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:39.968511 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5bc578-3ec1-4965-a6c8-fa29a73010bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:39.970762 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bc578-3ec1-4965-a6c8-fa29a73010bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:39.971242 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bc578-3ec1-4965-a6c8-fa29a73010bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:39.971707 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bc578-3ec1-4965-a6c8-fa29a73010bd admin admin] PATH_INFO: `/v3/projects/86f2720035c54fefb0c1d4bb47661361/users/12e2a78ac61d4749ab8e20647e009c85/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:39.974229 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5bc578-3ec1-4965-a6c8-fa29a73010bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=86f2720035c54fefb0c1d4bb47661361, user_id=12e2a78ac61d4749ab8e20647e009c85, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:39.977697 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e747a09e-1ba4-4513-8459-f40cfc3f4d77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:39.979523 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e747a09e-1ba4-4513-8459-f40cfc3f4d77 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:39.979802 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5bc578-3ec1-4965-a6c8-fa29a73010bd admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:40.010260 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 804/1619] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:40.013330 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747a09e-1ba4-4513-8459-f40cfc3f4d77 service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:40.023330 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 816/1620] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:39 2022] PUT /identity/v3/projects/86f2720035c54fefb0c1d4bb47661361/users/12e2a78ac61d4749ab8e20647e009c85/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:40.025419 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-e747a09e-1ba4-4513-8459-f40cfc3f4d77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:40.044790 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747a09e-1ba4-4513-8459-f40cfc3f4d77 service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:40.080240 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:40.142338 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63f4a9-eae6-4962-a0a1-13b980fda365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:40.143649 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63f4a9-eae6-4962-a0a1-13b980fda365 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:40.143982 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63f4a9-eae6-4962-a0a1-13b980fda365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:40.144310 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63f4a9-eae6-4962-a0a1-13b980fda365 admin admin] PATH_INFO: `/v3/users/cdcd5135201c4174b320fa07c1604f48` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:40.146097 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63f4a9-eae6-4962-a0a1-13b980fda365 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdcd5135201c4174b320fa07c1604f48)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:40.150945 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63f4a9-eae6-4962-a0a1-13b980fda365 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:40.278751 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-eb63f4a9-eae6-4962-a0a1-13b980fda365 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdcd5135201c4174b320fa07c1604f48'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:40.279612 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-eb63f4a9-eae6-4962-a0a1-13b980fda365 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdcd5135201c4174b320fa07c1604f48'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:40.293842 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-eb63f4a9-eae6-4962-a0a1-13b980fda365 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdcd5135201c4174b320fa07c1604f48'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:40.326295 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-eb63f4a9-eae6-4962-a0a1-13b980fda365 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdcd5135201c4174b320fa07c1604f48'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:40.344079 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-eb63f4a9-eae6-4962-a0a1-13b980fda365 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdcd5135201c4174b320fa07c1604f48'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:40.372958 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 805/1621] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:40 2022] DELETE /identity/v3/users/cdcd5135201c4174b320fa07c1604f48 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:41.106808 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5bc578-3ec1-4965-a6c8-fa29a73010bd admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:41.108628 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-fc5bc578-3ec1-4965-a6c8-fa29a73010bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:41.135431 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5bc578-3ec1-4965-a6c8-fa29a73010bd admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:41.253329 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ac62b-f749-4051-95d7-d5692320858e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:41.253987 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ac62b-f749-4051-95d7-d5692320858e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:41.254420 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ac62b-f749-4051-95d7-d5692320858e admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:41.254531 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ac62b-f749-4051-95d7-d5692320858e admin admin] PATH_INFO: `/v3/projects/d5cccf8559be41d688dbb5eabdf4a78c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:41.256105 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48ac62b-f749-4051-95d7-d5692320858e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5cccf8559be41d688dbb5eabdf4a78c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:41.258469 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48ac62b-f749-4051-95d7-d5692320858e admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:41.333912 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c48ac62b-f749-4051-95d7-d5692320858e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5cccf8559be41d688dbb5eabdf4a78c', 'request_id': 'req-c48ac62b-f749-4051-95d7-d5692320858e'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:41.346282 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c48ac62b-f749-4051-95d7-d5692320858e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5cccf8559be41d688dbb5eabdf4a78c', 'request_id': 'req-c48ac62b-f749-4051-95d7-d5692320858e'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:41.354597 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c48ac62b-f749-4051-95d7-d5692320858e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5cccf8559be41d688dbb5eabdf4a78c', 'request_id': 'req-c48ac62b-f749-4051-95d7-d5692320858e'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:41.372558 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c48ac62b-f749-4051-95d7-d5692320858e admin admin] The token cache is being invalidate because project d5cccf8559be41d688dbb5eabdf4a78c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:41.372900 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c48ac62b-f749-4051-95d7-d5692320858e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:41.373829 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c48ac62b-f749-4051-95d7-d5692320858e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:41.387384 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 817/1622] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:41 2022] DELETE /identity/v3/projects/d5cccf8559be41d688dbb5eabdf4a78c => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:41.387929 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87442cf2-90e7-44a8-b02e-b0dd293be1e7 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:41.388440 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87442cf2-90e7-44a8-b02e-b0dd293be1e7 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:41.388582 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87442cf2-90e7-44a8-b02e-b0dd293be1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:41.411305 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-87442cf2-90e7-44a8-b02e-b0dd293be1e7 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:41.412885 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-87442cf2-90e7-44a8-b02e-b0dd293be1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:41.494550 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 806/1623] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:41.497120 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd61c61-cbb7-48bb-98c0-9fa8fc29daf7 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:41.497419 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd61c61-cbb7-48bb-98c0-9fa8fc29daf7 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:41.497670 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd61c61-cbb7-48bb-98c0-9fa8fc29daf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:41.520700 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-5cd61c61-cbb7-48bb-98c0-9fa8fc29daf7 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:41.522215 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-5cd61c61-cbb7-48bb-98c0-9fa8fc29daf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:41.534736 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 818/1624] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:41.537263 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87442cf2-90e7-44a8-b02e-b0dd293be1e7 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:41.547781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87442cf2-90e7-44a8-b02e-b0dd293be1e7 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:41.552555 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0449b9-f791-48b7-a6ea-b2bdd56ba7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:41.553201 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0449b9-f791-48b7-a6ea-b2bdd56ba7f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:41.553475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0449b9-f791-48b7-a6ea-b2bdd56ba7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:41.553742 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0449b9-f791-48b7-a6ea-b2bdd56ba7f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:41.554396 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0449b9-f791-48b7-a6ea-b2bdd56ba7f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:41.554730 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0449b9-f791-48b7-a6ea-b2bdd56ba7f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:41.556397 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0449b9-f791-48b7-a6ea-b2bdd56ba7f1 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:41.570686 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 807/1625] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:28:41.574960 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd61c61-cbb7-48bb-98c0-9fa8fc29daf7 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:41.586085 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd61c61-cbb7-48bb-98c0-9fa8fc29daf7 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:41.591305 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7b9b3-57bb-4913-8df2-b79d14e8c4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:41.592746 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7b9b3-57bb-4913-8df2-b79d14e8c4ea admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:41.592746 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7b9b3-57bb-4913-8df2-b79d14e8c4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:41.592746 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7b9b3-57bb-4913-8df2-b79d14e8c4ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:41.593493 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c7b9b3-57bb-4913-8df2-b79d14e8c4ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:41.595872 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c7b9b3-57bb-4913-8df2-b79d14e8c4ea admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:41.603601 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:41.634718 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 819/1626] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:41 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:28:41.636372 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0449b9-f791-48b7-a6ea-b2bdd56ba7f1 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:41.647074 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0449b9-f791-48b7-a6ea-b2bdd56ba7f1 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:41.682265 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:41.737526 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7b9b3-57bb-4913-8df2-b79d14e8c4ea admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:41.739747 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-34c7b9b3-57bb-4913-8df2-b79d14e8c4ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:41.742087 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865fa8fa-81a8-4e0a-b7b1-b62561a1fc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:41.742854 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fa8fa-81a8-4e0a-b7b1-b62561a1fc7b admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:41.743222 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fa8fa-81a8-4e0a-b7b1-b62561a1fc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:41.743576 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fa8fa-81a8-4e0a-b7b1-b62561a1fc7b admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:41.744425 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865fa8fa-81a8-4e0a-b7b1-b62561a1fc7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:41.747093 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865fa8fa-81a8-4e0a-b7b1-b62561a1fc7b admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:41.757670 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7b9b3-57bb-4913-8df2-b79d14e8c4ea admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:41.766019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f66c78-e6e7-4221-9810-45386c385aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:41.766407 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f66c78-e6e7-4221-9810-45386c385aba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:41.766783 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f66c78-e6e7-4221-9810-45386c385aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:41.767007 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f66c78-e6e7-4221-9810-45386c385aba admin admin] PATH_INFO: `/v3/projects/2d30a300103c41a7a2e3a634c59d2a48` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:41.768419 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f66c78-e6e7-4221-9810-45386c385aba admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d30a300103c41a7a2e3a634c59d2a48)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:41.770547 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f66c78-e6e7-4221-9810-45386c385aba admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:41.791010 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 808/1627] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:41 2022] POST /identity/v3/users => generated 293 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:41.791010 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865fa8fa-81a8-4e0a-b7b1-b62561a1fc7b admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:41.822067 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865fa8fa-81a8-4e0a-b7b1-b62561a1fc7b admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:41.838959 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d0bd42-729a-4ca3-be11-6b08293828de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:41.839599 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0bd42-729a-4ca3-be11-6b08293828de admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:41.839876 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0bd42-729a-4ca3-be11-6b08293828de admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:41.840145 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0bd42-729a-4ca3-be11-6b08293828de admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:41.840933 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d0bd42-729a-4ca3-be11-6b08293828de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:41.842694 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d0bd42-729a-4ca3-be11-6b08293828de admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:41.864305 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 809/1628] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:41.868556 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d0bd42-729a-4ca3-be11-6b08293828de admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:41.888275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d0bd42-729a-4ca3-be11-6b08293828de admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:41.896779 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-95f66c78-e6e7-4221-9810-45386c385aba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d30a300103c41a7a2e3a634c59d2a48', 'request_id': 'req-95f66c78-e6e7-4221-9810-45386c385aba'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:41.918971 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-95f66c78-e6e7-4221-9810-45386c385aba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d30a300103c41a7a2e3a634c59d2a48', 'request_id': 'req-95f66c78-e6e7-4221-9810-45386c385aba'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:41.950482 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-95f66c78-e6e7-4221-9810-45386c385aba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d30a300103c41a7a2e3a634c59d2a48', 'request_id': 'req-95f66c78-e6e7-4221-9810-45386c385aba'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:41.970787 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca1d00a-c7b7-4117-bd03-f92324416a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:41.971534 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1d00a-c7b7-4117-bd03-f92324416a33 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:41.973018 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1d00a-c7b7-4117-bd03-f92324416a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:41.973312 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1d00a-c7b7-4117-bd03-f92324416a33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:41.974062 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca1d00a-c7b7-4117-bd03-f92324416a33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:41.979604 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-95f66c78-e6e7-4221-9810-45386c385aba admin admin] The token cache is being invalidate because project 2d30a300103c41a7a2e3a634c59d2a48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:41.980176 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-95f66c78-e6e7-4221-9810-45386c385aba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:41.981674 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-95f66c78-e6e7-4221-9810-45386c385aba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:41.986187 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 820/1629] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:41 2022] DELETE /identity/v3/projects/2d30a300103c41a7a2e3a634c59d2a48 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:41.988528 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca1d00a-c7b7-4117-bd03-f92324416a33 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:41.990523 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f66c78-e6e7-4221-9810-45386c385aba admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:41.993537 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-95f66c78-e6e7-4221-9810-45386c385aba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:42.013274 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 810/1630] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:28:42.016519 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca1d00a-c7b7-4117-bd03-f92324416a33 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:42.020007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-fca1d00a-c7b7-4117-bd03-f92324416a33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:42.024018 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f66c78-e6e7-4221-9810-45386c385aba admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:42.037281 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6a551-6d5f-45c8-9793-95c7714a0052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:42.037777 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6a551-6d5f-45c8-9793-95c7714a0052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:42.038053 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6a551-6d5f-45c8-9793-95c7714a0052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:42.039526 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6a551-6d5f-45c8-9793-95c7714a0052 admin admin] PATH_INFO: `/v3/users/65612a2e487348eea96254a3b6132ce0` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:42.041886 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a6a551-6d5f-45c8-9793-95c7714a0052 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65612a2e487348eea96254a3b6132ce0)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:42.044544 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca1d00a-c7b7-4117-bd03-f92324416a33 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:42.047111 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a6a551-6d5f-45c8-9793-95c7714a0052 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:42.056704 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea106ae-f8b7-4839-93f6-e9a55bd85c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:42.058064 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea106ae-f8b7-4839-93f6-e9a55bd85c27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:42.058372 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea106ae-f8b7-4839-93f6-e9a55bd85c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:42.058618 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea106ae-f8b7-4839-93f6-e9a55bd85c27 admin admin] PATH_INFO: `/v3/projects/0ba5cab8660f4702b4f40b19dd5f809d/users/275f7189ef0a4740a36fa399277fd725/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:42.110816 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea106ae-f8b7-4839-93f6-e9a55bd85c27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ba5cab8660f4702b4f40b19dd5f809d, user_id=275f7189ef0a4740a36fa399277fd725, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:42.110816 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea106ae-f8b7-4839-93f6-e9a55bd85c27 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:42.149764 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 811/1631] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:42 2022] PUT /identity/v3/projects/0ba5cab8660f4702b4f40b19dd5f809d/users/275f7189ef0a4740a36fa399277fd725/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:42.162242 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc19300-b617-4d5d-90c4-4e0cf38b4bff None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:42.162539 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc19300-b617-4d5d-90c4-4e0cf38b4bff None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:42.162821 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc19300-b617-4d5d-90c4-4e0cf38b4bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:42.174531 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-98a6a551-6d5f-45c8-9793-95c7714a0052 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65612a2e487348eea96254a3b6132ce0'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:42.205996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-8dc19300-b617-4d5d-90c4-4e0cf38b4bff None None] MFA Rules not processed for user `275f7189ef0a4740a36fa399277fd725`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:42.207535 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-8dc19300-b617-4d5d-90c4-4e0cf38b4bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:42.212298 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-98a6a551-6d5f-45c8-9793-95c7714a0052 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65612a2e487348eea96254a3b6132ce0'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:42.231144 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-98a6a551-6d5f-45c8-9793-95c7714a0052 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65612a2e487348eea96254a3b6132ce0'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:42.255151 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-98a6a551-6d5f-45c8-9793-95c7714a0052 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65612a2e487348eea96254a3b6132ce0'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:42.277440 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-98a6a551-6d5f-45c8-9793-95c7714a0052 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65612a2e487348eea96254a3b6132ce0'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:42.303880 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 821/1632] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:41 2022] DELETE /identity/v3/users/65612a2e487348eea96254a3b6132ce0 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:42.406767 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6a551-6d5f-45c8-9793-95c7714a0052 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:42.409342 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 812/1633] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:42.410889 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-98a6a551-6d5f-45c8-9793-95c7714a0052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:42.482808 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc19300-b617-4d5d-90c4-4e0cf38b4bff None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:42.484344 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-8dc19300-b617-4d5d-90c4-4e0cf38b4bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:42.498800 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=80,keystone:INSERT=3,keystone:DELETE=17 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:42.505464 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:42.530950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6a551-6d5f-45c8-9793-95c7714a0052 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:42.537015 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1063e0-f313-440d-bdea-578d9e4c10e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:42.538080 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1063e0-f313-440d-bdea-578d9e4c10e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:42.539217 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1063e0-f313-440d-bdea-578d9e4c10e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:42.539906 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1063e0-f313-440d-bdea-578d9e4c10e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:42.541419 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1063e0-f313-440d-bdea-578d9e4c10e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:42.544029 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1063e0-f313-440d-bdea-578d9e4c10e8 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:42.589596 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 822/1634] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:42 2022] POST /identity/v3/projects => generated 385 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:42.599596 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1063e0-f313-440d-bdea-578d9e4c10e8 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:42.611554 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc19300-b617-4d5d-90c4-4e0cf38b4bff None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:42.682417 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60347a8-497c-420d-9032-28d898d93e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:42.683090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60347a8-497c-420d-9032-28d898d93e18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:42.683360 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60347a8-497c-420d-9032-28d898d93e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:42.683648 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60347a8-497c-420d-9032-28d898d93e18 admin admin] PATH_INFO: `/v3/users/a2e2c74d31f94a498e3c2282dfeafe57` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:42.684786 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60347a8-497c-420d-9032-28d898d93e18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2e2c74d31f94a498e3c2282dfeafe57)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:42.686745 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60347a8-497c-420d-9032-28d898d93e18 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:42.688764 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1063e0-f313-440d-bdea-578d9e4c10e8 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:42.696831 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935aa39b-37a0-47c2-9b8e-3b6f8d70d4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:42.697732 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935aa39b-37a0-47c2-9b8e-3b6f8d70d4a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:42.698163 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935aa39b-37a0-47c2-9b8e-3b6f8d70d4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:42.698572 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935aa39b-37a0-47c2-9b8e-3b6f8d70d4a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:42.700462 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935aa39b-37a0-47c2-9b8e-3b6f8d70d4a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:42.704279 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935aa39b-37a0-47c2-9b8e-3b6f8d70d4a3 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:42.744856 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 823/1635] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:42 2022] POST /identity/v3/users => generated 299 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:42.749071 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935aa39b-37a0-47c2-9b8e-3b6f8d70d4a3 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:42.750786 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c60347a8-497c-420d-9032-28d898d93e18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2e2c74d31f94a498e3c2282dfeafe57'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:42.752111 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c60347a8-497c-420d-9032-28d898d93e18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2e2c74d31f94a498e3c2282dfeafe57'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:42.760519 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935aa39b-37a0-47c2-9b8e-3b6f8d70d4a3 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:42.761437 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c60347a8-497c-420d-9032-28d898d93e18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2e2c74d31f94a498e3c2282dfeafe57'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:42.766084 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf14ba3-991a-493e-b9ff-cd0b8a43bc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:42.766771 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf14ba3-991a-493e-b9ff-cd0b8a43bc89 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:42.767054 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf14ba3-991a-493e-b9ff-cd0b8a43bc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:42.767428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf14ba3-991a-493e-b9ff-cd0b8a43bc89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:42.768110 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf14ba3-991a-493e-b9ff-cd0b8a43bc89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:42.769860 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf14ba3-991a-493e-b9ff-cd0b8a43bc89 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:42.781892 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c60347a8-497c-420d-9032-28d898d93e18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2e2c74d31f94a498e3c2282dfeafe57'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:42.786699 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 824/1636] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:28:42.788666 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf14ba3-991a-493e-b9ff-cd0b8a43bc89 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:42.790862 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-2bf14ba3-991a-493e-b9ff-cd0b8a43bc89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:42.795153 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c60347a8-497c-420d-9032-28d898d93e18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2e2c74d31f94a498e3c2282dfeafe57'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:42.806782 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 813/1637] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:42 2022] DELETE /identity/v3/users/a2e2c74d31f94a498e3c2282dfeafe57 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:42.806967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf14ba3-991a-493e-b9ff-cd0b8a43bc89 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:42.812752 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78041dbc-8a84-4368-9296-92b4f5c34761 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:42.813326 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78041dbc-8a84-4368-9296-92b4f5c34761 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:42.813773 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78041dbc-8a84-4368-9296-92b4f5c34761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:42.844348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-78041dbc-8a84-4368-9296-92b4f5c34761 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:42.845886 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-78041dbc-8a84-4368-9296-92b4f5c34761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:42.898221 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aac9fa-6a90-465d-a7d1-eed1b4fe34b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:42.899135 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aac9fa-6a90-465d-a7d1-eed1b4fe34b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:42.899551 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aac9fa-6a90-465d-a7d1-eed1b4fe34b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:42.900005 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aac9fa-6a90-465d-a7d1-eed1b4fe34b1 admin admin] PATH_INFO: `/v3/projects/879377024bde46f38464aebc617e9c69/users/56d64cd6d7d44dc089cf25c00949504b/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:42.912470 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:42.922523 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aac9fa-6a90-465d-a7d1-eed1b4fe34b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=879377024bde46f38464aebc617e9c69, user_id=56d64cd6d7d44dc089cf25c00949504b, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:42.927256 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aac9fa-6a90-465d-a7d1-eed1b4fe34b1 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:42.956316 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 825/1638] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:42 2022] PUT /identity/v3/projects/879377024bde46f38464aebc617e9c69/users/56d64cd6d7d44dc089cf25c00949504b/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:42.958158 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a3f7f-6476-4ea0-9237-56ed6b4f2f8a None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:42.958447 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a3f7f-6476-4ea0-9237-56ed6b4f2f8a None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:42.961749 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a3f7f-6476-4ea0-9237-56ed6b4f2f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:42.988515 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 814/1639] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:42.989785 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78041dbc-8a84-4368-9296-92b4f5c34761 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:43.003877 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78041dbc-8a84-4368-9296-92b4f5c34761 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:43.037824 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-4b8a3f7f-6476-4ea0-9237-56ed6b4f2f8a None None] MFA Rules not processed for user `12e2a78ac61d4749ab8e20647e009c85`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:43.042438 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-4b8a3f7f-6476-4ea0-9237-56ed6b4f2f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:43.077791 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:43.106756 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662cccd-27b4-4ff8-a478-d78a4a2eba89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:43.107385 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662cccd-27b4-4ff8-a478-d78a4a2eba89 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:43.107662 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662cccd-27b4-4ff8-a478-d78a4a2eba89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:43.107928 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662cccd-27b4-4ff8-a478-d78a4a2eba89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:43.108577 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1662cccd-27b4-4ff8-a478-d78a4a2eba89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:43.111288 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1662cccd-27b4-4ff8-a478-d78a4a2eba89 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:43.128850 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 815/1640] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:43.128850 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662cccd-27b4-4ff8-a478-d78a4a2eba89 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:43.129807 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-1662cccd-27b4-4ff8-a478-d78a4a2eba89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:43.192712 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 826/1641] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:43.194497 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a3f7f-6476-4ea0-9237-56ed6b4f2f8a None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:43.196354 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-4b8a3f7f-6476-4ea0-9237-56ed6b4f2f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:43.215724 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a3f7f-6476-4ea0-9237-56ed6b4f2f8a None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:43.221582 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b585b-27bb-4be0-afa3-d0dc12c679cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:43.222633 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b585b-27bb-4be0-afa3-d0dc12c679cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:43.223044 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b585b-27bb-4be0-afa3-d0dc12c679cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:43.223618 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b585b-27bb-4be0-afa3-d0dc12c679cb admin admin] PATH_INFO: `/v3/projects/52940055c035479583438fac39e541b2` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:43.225584 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b585b-27bb-4be0-afa3-d0dc12c679cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=52940055c035479583438fac39e541b2)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:43.227924 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662cccd-27b4-4ff8-a478-d78a4a2eba89 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:43.228059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b585b-27bb-4be0-afa3-d0dc12c679cb admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:43.232149 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785baca8-e450-46b1-86a9-3acd504eacab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:43.232769 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785baca8-e450-46b1-86a9-3acd504eacab service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:43.233102 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785baca8-e450-46b1-86a9-3acd504eacab service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:43.233752 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785baca8-e450-46b1-86a9-3acd504eacab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:43.245707 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785baca8-e450-46b1-86a9-3acd504eacab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:43.247304 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785baca8-e450-46b1-86a9-3acd504eacab service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:43.259783 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 816/1642] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:43.260710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785baca8-e450-46b1-86a9-3acd504eacab service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:43.277403 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785baca8-e450-46b1-86a9-3acd504eacab service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:43.286871 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aca63c-5984-48ba-b265-0556cba7f299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:43.287559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aca63c-5984-48ba-b265-0556cba7f299 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:43.287837 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aca63c-5984-48ba-b265-0556cba7f299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:43.288115 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aca63c-5984-48ba-b265-0556cba7f299 admin admin] PATH_INFO: `/v3/projects/879377024bde46f38464aebc617e9c69/users/56d64cd6d7d44dc089cf25c00949504b/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:43.294386 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aca63c-5984-48ba-b265-0556cba7f299 admin admin] RBAC: Authorizing `identity:create_grant(project_id=879377024bde46f38464aebc617e9c69, user_id=56d64cd6d7d44dc089cf25c00949504b, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:43.297529 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aca63c-5984-48ba-b265-0556cba7f299 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:43.309647 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-a34b585b-27bb-4be0-afa3-d0dc12c679cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52940055c035479583438fac39e541b2', 'request_id': 'req-a34b585b-27bb-4be0-afa3-d0dc12c679cb'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:43.311042 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 817/1643] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:43 2022] PUT /identity/v3/projects/879377024bde46f38464aebc617e9c69/users/56d64cd6d7d44dc089cf25c00949504b/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:43.314244 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aca63c-5984-48ba-b265-0556cba7f299 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:43.321343 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-d2aca63c-5984-48ba-b265-0556cba7f299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:43.333822 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-a34b585b-27bb-4be0-afa3-d0dc12c679cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52940055c035479583438fac39e541b2', 'request_id': 'req-a34b585b-27bb-4be0-afa3-d0dc12c679cb'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:43.347504 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-a34b585b-27bb-4be0-afa3-d0dc12c679cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52940055c035479583438fac39e541b2', 'request_id': 'req-a34b585b-27bb-4be0-afa3-d0dc12c679cb'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:43.372836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-a34b585b-27bb-4be0-afa3-d0dc12c679cb admin admin] The token cache is being invalidate because project 52940055c035479583438fac39e541b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:43.373256 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-a34b585b-27bb-4be0-afa3-d0dc12c679cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:43.373992 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-a34b585b-27bb-4be0-afa3-d0dc12c679cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:43.375778 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 827/1644] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:43 2022] DELETE /identity/v3/projects/52940055c035479583438fac39e541b2 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:43.377558 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb9ce9-eaf5-4736-8a83-e964aae082ae None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:43.378024 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb9ce9-eaf5-4736-8a83-e964aae082ae None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:43.378494 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb9ce9-eaf5-4736-8a83-e964aae082ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:43.431594 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-becb9ce9-eaf5-4736-8a83-e964aae082ae None None] MFA Rules not processed for user `56d64cd6d7d44dc089cf25c00949504b`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:43.438019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-becb9ce9-eaf5-4736-8a83-e964aae082ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:43.443052 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aca63c-5984-48ba-b265-0556cba7f299 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:43.451842 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff8a9fc-b8a6-4013-ae3b-ec46e586697b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:43.452731 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff8a9fc-b8a6-4013-ae3b-ec46e586697b service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:43.454612 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff8a9fc-b8a6-4013-ae3b-ec46e586697b service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:43.455060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff8a9fc-b8a6-4013-ae3b-ec46e586697b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:43.464355 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-cff8a9fc-b8a6-4013-ae3b-ec46e586697b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:43.529151 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:43.591377 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 828/1645] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:43.592614 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb9ce9-eaf5-4736-8a83-e964aae082ae None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:43.595747 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-becb9ce9-eaf5-4736-8a83-e964aae082ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:43.640463 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff8a9fc-b8a6-4013-ae3b-ec46e586697b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:43.642151 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff8a9fc-b8a6-4013-ae3b-ec46e586697b service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:43.651853 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:43.671361 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 818/1646] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:43.715999 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb9ce9-eaf5-4736-8a83-e964aae082ae None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:43.722352 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9fc50-c988-4713-be32-9a39fe3d5ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:43.724588 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9fc50-c988-4713-be32-9a39fe3d5ed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:43.725131 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9fc50-c988-4713-be32-9a39fe3d5ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:43.725690 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9fc50-c988-4713-be32-9a39fe3d5ed2 admin admin] PATH_INFO: `/v3/users/47082dff733d4dd68dddf1dbd435f084` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:43.728456 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f9fc50-c988-4713-be32-9a39fe3d5ed2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47082dff733d4dd68dddf1dbd435f084)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:43.731406 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f9fc50-c988-4713-be32-9a39fe3d5ed2 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:43.745774 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0555e96-138d-426c-9321-73a083b299fc None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:43.746089 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0555e96-138d-426c-9321-73a083b299fc None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:43.746319 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0555e96-138d-426c-9321-73a083b299fc None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:43.749351 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 819/1647] 10.222.0.30 () {54 vars in 921 bytes} [Wed Nov 2 07:28:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:28:43.760585 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b63bef-8769-46e8-949e-f9c8145c9019 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:43.760734 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b63bef-8769-46e8-949e-f9c8145c9019 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:43.760950 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b63bef-8769-46e8-949e-f9c8145c9019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:43.816627 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-f2b63bef-8769-46e8-949e-f9c8145c9019 None None] MFA Rules not processed for user `e6495385dccc4fa48c05b1feb957846c`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:43.818521 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-f2b63bef-8769-46e8-949e-f9c8145c9019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:43.849353 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-56f9fc50-c988-4713-be32-9a39fe3d5ed2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47082dff733d4dd68dddf1dbd435f084'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:43.859688 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-56f9fc50-c988-4713-be32-9a39fe3d5ed2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47082dff733d4dd68dddf1dbd435f084'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:43.860920 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-56f9fc50-c988-4713-be32-9a39fe3d5ed2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47082dff733d4dd68dddf1dbd435f084'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:43.887938 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-56f9fc50-c988-4713-be32-9a39fe3d5ed2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47082dff733d4dd68dddf1dbd435f084'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:43.900722 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-56f9fc50-c988-4713-be32-9a39fe3d5ed2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47082dff733d4dd68dddf1dbd435f084'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:43.912005 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 829/1648] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:43 2022] DELETE /identity/v3/users/47082dff733d4dd68dddf1dbd435f084 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:43.988175 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 820/1649] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Nov 2 07:28:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:44.000686 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9fc50-c988-4713-be32-9a39fe3d5ed2 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:44.002672 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-56f9fc50-c988-4713-be32-9a39fe3d5ed2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:44.068938 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b63bef-8769-46e8-949e-f9c8145c9019 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:44.069956 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-f2b63bef-8769-46e8-949e-f9c8145c9019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:44.096604 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b63bef-8769-46e8-949e-f9c8145c9019 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:44.194177 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9fc50-c988-4713-be32-9a39fe3d5ed2 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:44.211627 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553cbc1-ca4f-4381-aaf6-3346173ff7b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:44.212307 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553cbc1-ca4f-4381-aaf6-3346173ff7b0 service nova] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:44.212677 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553cbc1-ca4f-4381-aaf6-3346173ff7b0 service nova] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:44.213207 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553cbc1-ca4f-4381-aaf6-3346173ff7b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:44.219253 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30259c03-54c5-4818-ad40-48a5af0d0875 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:44.219752 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30259c03-54c5-4818-ad40-48a5af0d0875 service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:44.220029 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30259c03-54c5-4818-ad40-48a5af0d0875 service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:44.220458 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30259c03-54c5-4818-ad40-48a5af0d0875 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:44.222320 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-30259c03-54c5-4818-ad40-48a5af0d0875 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:44.242745 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f553cbc1-ca4f-4381-aaf6-3346173ff7b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:44.247565 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f553cbc1-ca4f-4381-aaf6-3346173ff7b0 service nova] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:44.276419 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 830/1650] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:44.357295 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30259c03-54c5-4818-ad40-48a5af0d0875 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:44.359661 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30259c03-54c5-4818-ad40-48a5af0d0875 service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:44.415434 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 821/1651] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:44 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:44.476042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553cbc1-ca4f-4381-aaf6-3346173ff7b0 service nova] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:44.478950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-f553cbc1-ca4f-4381-aaf6-3346173ff7b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:44.567711 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:44.631725 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553cbc1-ca4f-4381-aaf6-3346173ff7b0 service nova] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:44.639169 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61777f0c-90e9-4476-89e2-eed29a79f564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:44.639822 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61777f0c-90e9-4476-89e2-eed29a79f564 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:44.640093 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61777f0c-90e9-4476-89e2-eed29a79f564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:44.640370 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61777f0c-90e9-4476-89e2-eed29a79f564 admin admin] PATH_INFO: `/v3/projects/d5d3495a3b0e4b91bc389964ef808847` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:44.641494 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61777f0c-90e9-4476-89e2-eed29a79f564 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5d3495a3b0e4b91bc389964ef808847)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:44.643455 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61777f0c-90e9-4476-89e2-eed29a79f564 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:44.703575 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-61777f0c-90e9-4476-89e2-eed29a79f564 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5d3495a3b0e4b91bc389964ef808847', 'request_id': 'req-61777f0c-90e9-4476-89e2-eed29a79f564'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:44.732098 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-61777f0c-90e9-4476-89e2-eed29a79f564 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5d3495a3b0e4b91bc389964ef808847', 'request_id': 'req-61777f0c-90e9-4476-89e2-eed29a79f564'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:44.737159 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30259c03-54c5-4818-ad40-48a5af0d0875 service cinder] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:44.738796 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-30259c03-54c5-4818-ad40-48a5af0d0875 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:44.746924 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-61777f0c-90e9-4476-89e2-eed29a79f564 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5d3495a3b0e4b91bc389964ef808847', 'request_id': 'req-61777f0c-90e9-4476-89e2-eed29a79f564'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:44.749409 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:44.761837 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30259c03-54c5-4818-ad40-48a5af0d0875 service cinder] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:44.768917 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-61777f0c-90e9-4476-89e2-eed29a79f564 admin admin] The token cache is being invalidate because project d5d3495a3b0e4b91bc389964ef808847 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:44.768917 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-61777f0c-90e9-4476-89e2-eed29a79f564 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:44.769850 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-61777f0c-90e9-4476-89e2-eed29a79f564 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:44.773446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 831/1652] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:44 2022] DELETE /identity/v3/projects/d5d3495a3b0e4b91bc389964ef808847 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:44.780294 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1211421-b587-48d9-97d3-d0e20ad54bce None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:44.780576 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1211421-b587-48d9-97d3-d0e20ad54bce None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:44.780935 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1211421-b587-48d9-97d3-d0e20ad54bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:44.804191 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-e1211421-b587-48d9-97d3-d0e20ad54bce None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:44.805428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-e1211421-b587-48d9-97d3-d0e20ad54bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:44.858151 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd482971-4ac8-4376-bf59-aba9867fd0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:44.858832 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd482971-4ac8-4376-bf59-aba9867fd0f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:44.859105 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd482971-4ac8-4376-bf59-aba9867fd0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:44.859366 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd482971-4ac8-4376-bf59-aba9867fd0f2 admin admin] PATH_INFO: `/v3/projects/1dadb9283d68453b8987c450c6cfb79e` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:44.860869 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd482971-4ac8-4376-bf59-aba9867fd0f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dadb9283d68453b8987c450c6cfb79e)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:44.862835 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd482971-4ac8-4376-bf59-aba9867fd0f2 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:44.873173 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 832/1653] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:44.884353 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2c60f-0051-4e5a-9b1c-2ec49fffc863 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:44.884794 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2c60f-0051-4e5a-9b1c-2ec49fffc863 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:44.885069 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2c60f-0051-4e5a-9b1c-2ec49fffc863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:44.906618 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-95e2c60f-0051-4e5a-9b1c-2ec49fffc863 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:44.907984 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-95e2c60f-0051-4e5a-9b1c-2ec49fffc863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:44.922485 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-bd482971-4ac8-4376-bf59-aba9867fd0f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dadb9283d68453b8987c450c6cfb79e', 'request_id': 'req-bd482971-4ac8-4376-bf59-aba9867fd0f2'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:44.934572 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-bd482971-4ac8-4376-bf59-aba9867fd0f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dadb9283d68453b8987c450c6cfb79e', 'request_id': 'req-bd482971-4ac8-4376-bf59-aba9867fd0f2'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:44.941984 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-bd482971-4ac8-4376-bf59-aba9867fd0f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dadb9283d68453b8987c450c6cfb79e', 'request_id': 'req-bd482971-4ac8-4376-bf59-aba9867fd0f2'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:44.957866 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-bd482971-4ac8-4376-bf59-aba9867fd0f2 admin admin] The token cache is being invalidate because project 1dadb9283d68453b8987c450c6cfb79e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:44.958155 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-bd482971-4ac8-4376-bf59-aba9867fd0f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:44.958821 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-bd482971-4ac8-4376-bf59-aba9867fd0f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:44.964325 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 822/1654] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:44 2022] DELETE /identity/v3/projects/1dadb9283d68453b8987c450c6cfb79e => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:44.968183 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd482971-4ac8-4376-bf59-aba9867fd0f2 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:44.969557 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-bd482971-4ac8-4376-bf59-aba9867fd0f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:44.976339 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:44.990349 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 833/1655] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:44.995347 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2c60f-0051-4e5a-9b1c-2ec49fffc863 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:45.006060 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2c60f-0051-4e5a-9b1c-2ec49fffc863 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:45.012338 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164dc2e-1d52-4a1c-a892-35f146de2a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:45.013226 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164dc2e-1d52-4a1c-a892-35f146de2a0d admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:45.013510 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164dc2e-1d52-4a1c-a892-35f146de2a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:45.013783 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164dc2e-1d52-4a1c-a892-35f146de2a0d admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:45.015343 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b164dc2e-1d52-4a1c-a892-35f146de2a0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:45.015652 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b164dc2e-1d52-4a1c-a892-35f146de2a0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:45.017376 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b164dc2e-1d52-4a1c-a892-35f146de2a0d admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:45.039270 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 834/1656] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:28:45.044342 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164dc2e-1d52-4a1c-a892-35f146de2a0d admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:45.058433 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164dc2e-1d52-4a1c-a892-35f146de2a0d admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:45.073393 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58efe8cf-4d2f-4455-b576-f0743cd0175c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:45.074737 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58efe8cf-4d2f-4455-b576-f0743cd0175c admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:45.075026 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58efe8cf-4d2f-4455-b576-f0743cd0175c admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:45.077428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58efe8cf-4d2f-4455-b576-f0743cd0175c admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:45.077802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd482971-4ac8-4376-bf59-aba9867fd0f2 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:45.078122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58efe8cf-4d2f-4455-b576-f0743cd0175c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:45.082873 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe372fa2-44cd-4319-bbcf-3ed616f17d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:45.083911 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe372fa2-44cd-4319-bbcf-3ed616f17d8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:45.084375 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe372fa2-44cd-4319-bbcf-3ed616f17d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:45.084713 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe372fa2-44cd-4319-bbcf-3ed616f17d8f admin admin] PATH_INFO: `/v3/users/c4370cfb4d894d7fb1e96a03cbeffb46` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:45.085924 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe372fa2-44cd-4319-bbcf-3ed616f17d8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4370cfb4d894d7fb1e96a03cbeffb46)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:45.088032 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe372fa2-44cd-4319-bbcf-3ed616f17d8f admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:45.092281 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58efe8cf-4d2f-4455-b576-f0743cd0175c admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:45.124896 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 835/1657] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:45 2022] POST /identity/v3/projects => generated 373 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:45.129820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58efe8cf-4d2f-4455-b576-f0743cd0175c admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:45.154756 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:45.170961 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-fe372fa2-44cd-4319-bbcf-3ed616f17d8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4370cfb4d894d7fb1e96a03cbeffb46'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:45.171528 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-fe372fa2-44cd-4319-bbcf-3ed616f17d8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4370cfb4d894d7fb1e96a03cbeffb46'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:45.182340 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-fe372fa2-44cd-4319-bbcf-3ed616f17d8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4370cfb4d894d7fb1e96a03cbeffb46'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:45.198974 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-fe372fa2-44cd-4319-bbcf-3ed616f17d8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4370cfb4d894d7fb1e96a03cbeffb46'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:45.209216 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-fe372fa2-44cd-4319-bbcf-3ed616f17d8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4370cfb4d894d7fb1e96a03cbeffb46'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:45.220380 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 823/1658] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:44 2022] DELETE /identity/v3/users/c4370cfb4d894d7fb1e96a03cbeffb46 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:45.220559 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58efe8cf-4d2f-4455-b576-f0743cd0175c admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:45.220991 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe372fa2-44cd-4319-bbcf-3ed616f17d8f admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:45.220991 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-fe372fa2-44cd-4319-bbcf-3ed616f17d8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:45.301484 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd65699-6a5a-46dc-867b-13bd61bd1caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:45.302412 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd65699-6a5a-46dc-867b-13bd61bd1caf admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:45.302869 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd65699-6a5a-46dc-867b-13bd61bd1caf admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:45.303227 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd65699-6a5a-46dc-867b-13bd61bd1caf admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:45.303975 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd65699-6a5a-46dc-867b-13bd61bd1caf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:45.306546 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd65699-6a5a-46dc-867b-13bd61bd1caf admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:45.341586 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 836/1659] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:45 2022] POST /identity/v3/users => generated 287 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:45.345312 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd65699-6a5a-46dc-867b-13bd61bd1caf admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:45.345793 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe372fa2-44cd-4319-bbcf-3ed616f17d8f admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:45.348432 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-ebd65699-6a5a-46dc-867b-13bd61bd1caf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:45.359143 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407205b-a190-458e-8a51-aec765e52db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:45.359808 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407205b-a190-458e-8a51-aec765e52db8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:45.360080 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407205b-a190-458e-8a51-aec765e52db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:45.360346 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407205b-a190-458e-8a51-aec765e52db8 admin admin] PATH_INFO: `/v3/users/275f7189ef0a4740a36fa399277fd725` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:45.362740 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8407205b-a190-458e-8a51-aec765e52db8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=275f7189ef0a4740a36fa399277fd725)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:45.364719 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8407205b-a190-458e-8a51-aec765e52db8 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:45.373048 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd65699-6a5a-46dc-867b-13bd61bd1caf admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:45.395023 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b11b2-4ef3-49c6-8f18-1797b7bd2a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:45.395023 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b11b2-4ef3-49c6-8f18-1797b7bd2a41 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:45.395023 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b11b2-4ef3-49c6-8f18-1797b7bd2a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:45.395023 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b11b2-4ef3-49c6-8f18-1797b7bd2a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:45.395320 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b11b2-4ef3-49c6-8f18-1797b7bd2a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:45.395320 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b11b2-4ef3-49c6-8f18-1797b7bd2a41 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:45.462313 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 837/1660] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:45.463936 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b11b2-4ef3-49c6-8f18-1797b7bd2a41 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:45.494456 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b11b2-4ef3-49c6-8f18-1797b7bd2a41 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:45.509720 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d316d4f-2868-4792-bbad-38de44b4044f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:45.510367 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d316d4f-2868-4792-bbad-38de44b4044f admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:45.510960 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d316d4f-2868-4792-bbad-38de44b4044f admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:45.511237 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d316d4f-2868-4792-bbad-38de44b4044f admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:45.511892 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d316d4f-2868-4792-bbad-38de44b4044f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:45.513506 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d316d4f-2868-4792-bbad-38de44b4044f admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:45.520132 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-8407205b-a190-458e-8a51-aec765e52db8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '275f7189ef0a4740a36fa399277fd725'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:45.521162 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-8407205b-a190-458e-8a51-aec765e52db8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '275f7189ef0a4740a36fa399277fd725'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:45.529673 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-8407205b-a190-458e-8a51-aec765e52db8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '275f7189ef0a4740a36fa399277fd725'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:45.544613 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-8407205b-a190-458e-8a51-aec765e52db8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '275f7189ef0a4740a36fa399277fd725'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:45.561134 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 838/1661] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:45.564266 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d316d4f-2868-4792-bbad-38de44b4044f admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:45.568556 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-7d316d4f-2868-4792-bbad-38de44b4044f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:45.569135 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-8407205b-a190-458e-8a51-aec765e52db8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '275f7189ef0a4740a36fa399277fd725'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:45.582555 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 824/1662] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:45 2022] DELETE /identity/v3/users/275f7189ef0a4740a36fa399277fd725 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:45.583908 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407205b-a190-458e-8a51-aec765e52db8 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:45.587471 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-8407205b-a190-458e-8a51-aec765e52db8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:45.668253 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:45.687229 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:45.701746 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d316d4f-2868-4792-bbad-38de44b4044f admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:45.749472 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407205b-a190-458e-8a51-aec765e52db8 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:45.756261 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcafb654-4bfe-43ef-9ce2-54052918048d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:45.758509 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcafb654-4bfe-43ef-9ce2-54052918048d admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:45.758509 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcafb654-4bfe-43ef-9ce2-54052918048d admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:45.758509 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcafb654-4bfe-43ef-9ce2-54052918048d admin admin] PATH_INFO: `/v3/projects/042319c18f6c49fbb748edfe946d1399/users/be71c2333f684d91ac423548e450b5ee/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:45.776676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcafb654-4bfe-43ef-9ce2-54052918048d admin admin] RBAC: Authorizing `identity:create_grant(project_id=042319c18f6c49fbb748edfe946d1399, user_id=be71c2333f684d91ac423548e450b5ee, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:45.779860 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcafb654-4bfe-43ef-9ce2-54052918048d admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:45.801407 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff62c8-212c-4c0b-b0b6-5c029ed7a5b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:45.802051 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff62c8-212c-4c0b-b0b6-5c029ed7a5b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:45.802328 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff62c8-212c-4c0b-b0b6-5c029ed7a5b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:45.802600 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff62c8-212c-4c0b-b0b6-5c029ed7a5b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:45.804681 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-a8ff62c8-212c-4c0b-b0b6-5c029ed7a5b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:45.805859 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 825/1663] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:45 2022] PUT /identity/v3/projects/042319c18f6c49fbb748edfe946d1399/users/be71c2333f684d91ac423548e450b5ee/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:45.807680 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94383b0f-74eb-4163-b6b5-f814c213e449 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:45.808028 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94383b0f-74eb-4163-b6b5-f814c213e449 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:45.808678 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94383b0f-74eb-4163-b6b5-f814c213e449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:45.834551 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-94383b0f-74eb-4163-b6b5-f814c213e449 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:45.836080 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-94383b0f-74eb-4163-b6b5-f814c213e449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:45.906171 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff62c8-212c-4c0b-b0b6-5c029ed7a5b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:45.908436 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff62c8-212c-4c0b-b0b6-5c029ed7a5b2 service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:45.924596 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 826/1664] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:45.925612 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94383b0f-74eb-4163-b6b5-f814c213e449 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:45.927054 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-94383b0f-74eb-4163-b6b5-f814c213e449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:45.936080 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 839/1665] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:45 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:45.941291 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94383b0f-74eb-4163-b6b5-f814c213e449 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:45.948867 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e517a-56e8-4812-bb4a-964ee3fc1e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:45.950116 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e517a-56e8-4812-bb4a-964ee3fc1e8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:45.950476 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e517a-56e8-4812-bb4a-964ee3fc1e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:45.950825 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e517a-56e8-4812-bb4a-964ee3fc1e8a admin admin] PATH_INFO: `/v3/projects/d7029a72e6c14fc3bb9622f277c2b093` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:45.953242 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67e517a-56e8-4812-bb4a-964ee3fc1e8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7029a72e6c14fc3bb9622f277c2b093)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:45.954111 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87e46b-5a41-45be-983f-14ae8120380f None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:45.954405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87e46b-5a41-45be-983f-14ae8120380f None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:45.954709 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87e46b-5a41-45be-983f-14ae8120380f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:45.955517 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67e517a-56e8-4812-bb4a-964ee3fc1e8a admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:46.024730 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-8d87e46b-5a41-45be-983f-14ae8120380f None None] MFA Rules not processed for user `be71c2333f684d91ac423548e450b5ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:46.026053 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-8d87e46b-5a41-45be-983f-14ae8120380f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:46.039793 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c67e517a-56e8-4812-bb4a-964ee3fc1e8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7029a72e6c14fc3bb9622f277c2b093', 'request_id': 'req-c67e517a-56e8-4812-bb4a-964ee3fc1e8a'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:46.050502 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c67e517a-56e8-4812-bb4a-964ee3fc1e8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7029a72e6c14fc3bb9622f277c2b093', 'request_id': 'req-c67e517a-56e8-4812-bb4a-964ee3fc1e8a'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:46.066138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c67e517a-56e8-4812-bb4a-964ee3fc1e8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7029a72e6c14fc3bb9622f277c2b093', 'request_id': 'req-c67e517a-56e8-4812-bb4a-964ee3fc1e8a'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:46.096541 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c67e517a-56e8-4812-bb4a-964ee3fc1e8a admin admin] The token cache is being invalidate because project d7029a72e6c14fc3bb9622f277c2b093 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:46.096876 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c67e517a-56e8-4812-bb4a-964ee3fc1e8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:46.098181 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c67e517a-56e8-4812-bb4a-964ee3fc1e8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:46.099875 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 827/1666] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:45 2022] DELETE /identity/v3/projects/d7029a72e6c14fc3bb9622f277c2b093 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:46.100657 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e517a-56e8-4812-bb4a-964ee3fc1e8a admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:46.110739 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-c67e517a-56e8-4812-bb4a-964ee3fc1e8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:46.146571 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 840/1667] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:46.152610 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86396ac8-f0a8-458d-b0bb-93a99f7906ca None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:46.152896 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86396ac8-f0a8-458d-b0bb-93a99f7906ca None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:46.153237 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86396ac8-f0a8-458d-b0bb-93a99f7906ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:46.179988 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-86396ac8-f0a8-458d-b0bb-93a99f7906ca None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:46.182613 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-86396ac8-f0a8-458d-b0bb-93a99f7906ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:46.186635 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:46.218216 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e517a-56e8-4812-bb4a-964ee3fc1e8a admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:46.225796 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b9f14-2466-4a9a-b639-f82c20ffc542 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:46.226898 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b9f14-2466-4a9a-b639-f82c20ffc542 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:46.227250 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b9f14-2466-4a9a-b639-f82c20ffc542 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:46.227527 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b9f14-2466-4a9a-b639-f82c20ffc542 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:46.229927 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-bc3b9f14-2466-4a9a-b639-f82c20ffc542 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:46.277537 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:46.317168 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 841/1668] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:46.318007 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86396ac8-f0a8-458d-b0bb-93a99f7906ca None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:46.319532 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3b9f14-2466-4a9a-b639-f82c20ffc542 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:46.321747 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3b9f14-2466-4a9a-b639-f82c20ffc542 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:46.326822 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-86396ac8-f0a8-458d-b0bb-93a99f7906ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:46.346397 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 828/1669] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:46.347686 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441fe6e-530b-49dc-801f-56d7f1f3e67f None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:46.348036 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441fe6e-530b-49dc-801f-56d7f1f3e67f None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:46.348377 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441fe6e-530b-49dc-801f-56d7f1f3e67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:46.372218 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-0441fe6e-530b-49dc-801f-56d7f1f3e67f None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:46.374108 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-0441fe6e-530b-49dc-801f-56d7f1f3e67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:46.383171 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 829/1670] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:46.387946 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0441fe6e-530b-49dc-801f-56d7f1f3e67f None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:46.399398 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0441fe6e-530b-49dc-801f-56d7f1f3e67f None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:46.406383 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a0b0b-831e-4f2d-ad90-b3e03e20b2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:46.407286 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a0b0b-831e-4f2d-ad90-b3e03e20b2cd admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:46.407772 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a0b0b-831e-4f2d-ad90-b3e03e20b2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:46.408671 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a0b0b-831e-4f2d-ad90-b3e03e20b2cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:46.410093 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a0b0b-831e-4f2d-ad90-b3e03e20b2cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:46.410093 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a0b0b-831e-4f2d-ad90-b3e03e20b2cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:46.411948 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a0b0b-831e-4f2d-ad90-b3e03e20b2cd admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:46.433651 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 830/1671] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:28:46.435105 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a0b0b-831e-4f2d-ad90-b3e03e20b2cd admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:46.436140 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86396ac8-f0a8-458d-b0bb-93a99f7906ca None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:46.442889 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035be16d-f2b4-4330-b3de-f1299767bd64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:46.443593 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035be16d-f2b4-4330-b3de-f1299767bd64 service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:46.443924 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035be16d-f2b4-4330-b3de-f1299767bd64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:46.444200 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035be16d-f2b4-4330-b3de-f1299767bd64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:46.446898 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a0b0b-831e-4f2d-ad90-b3e03e20b2cd admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:46.453871 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b6cdc-fce9-476b-ad1b-413c279f7e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:46.455294 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b6cdc-fce9-476b-ad1b-413c279f7e8a admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:46.455641 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b6cdc-fce9-476b-ad1b-413c279f7e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:46.456153 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b6cdc-fce9-476b-ad1b-413c279f7e8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:46.456925 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b6cdc-fce9-476b-ad1b-413c279f7e8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:46.458185 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035be16d-f2b4-4330-b3de-f1299767bd64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:46.459529 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b6cdc-fce9-476b-ad1b-413c279f7e8a admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:46.460402 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035be16d-f2b4-4330-b3de-f1299767bd64 service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:46.475302 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 842/1672] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:46.503896 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035be16d-f2b4-4330-b3de-f1299767bd64 service cinder] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:46.505308 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-035be16d-f2b4-4330-b3de-f1299767bd64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:46.509960 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 831/1673] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:46 2022] POST /identity/v3/projects => generated 379 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:46.510871 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b6cdc-fce9-476b-ad1b-413c279f7e8a admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:46.520655 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035be16d-f2b4-4330-b3de-f1299767bd64 service cinder] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:46.604814 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b6cdc-fce9-476b-ad1b-413c279f7e8a admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:46.610448 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5a4fd-c30e-470e-8225-beaa55b7f30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:46.611584 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5a4fd-c30e-470e-8225-beaa55b7f30b admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:46.612007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5a4fd-c30e-470e-8225-beaa55b7f30b admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:46.612284 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5a4fd-c30e-470e-8225-beaa55b7f30b admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:46.612978 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5a4fd-c30e-470e-8225-beaa55b7f30b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:46.615003 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974311b1-26c4-4cdb-815d-b9305794a017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:46.615134 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5a4fd-c30e-470e-8225-beaa55b7f30b admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:46.615792 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974311b1-26c4-4cdb-815d-b9305794a017 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:46.616076 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974311b1-26c4-4cdb-815d-b9305794a017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:46.616353 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974311b1-26c4-4cdb-815d-b9305794a017 admin admin] PATH_INFO: `/v3/users/be71c2333f684d91ac423548e450b5ee` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:46.617951 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974311b1-26c4-4cdb-815d-b9305794a017 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be71c2333f684d91ac423548e450b5ee)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:46.620866 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974311b1-26c4-4cdb-815d-b9305794a017 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:46.647312 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 832/1674] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:46 2022] POST /identity/v3/users => generated 290 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:46.651732 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5a4fd-c30e-470e-8225-beaa55b7f30b admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:46.665131 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5a4fd-c30e-470e-8225-beaa55b7f30b admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:46.672076 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782bc108-de32-4c44-8185-e9d29903e18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:46.672990 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782bc108-de32-4c44-8185-e9d29903e18b admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:46.673434 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782bc108-de32-4c44-8185-e9d29903e18b admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:46.673889 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782bc108-de32-4c44-8185-e9d29903e18b admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:46.674776 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782bc108-de32-4c44-8185-e9d29903e18b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:46.677005 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782bc108-de32-4c44-8185-e9d29903e18b admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:46.693799 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 833/1675] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:46.694958 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782bc108-de32-4c44-8185-e9d29903e18b admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:46.700448 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-974311b1-26c4-4cdb-815d-b9305794a017 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be71c2333f684d91ac423548e450b5ee'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:46.708426 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782bc108-de32-4c44-8185-e9d29903e18b admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:46.726191 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-974311b1-26c4-4cdb-815d-b9305794a017 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be71c2333f684d91ac423548e450b5ee'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:46.727702 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-974311b1-26c4-4cdb-815d-b9305794a017 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be71c2333f684d91ac423548e450b5ee'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:46.729149 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd25fab-112b-4431-907e-7fb576d9af45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:46.729837 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd25fab-112b-4431-907e-7fb576d9af45 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:46.730081 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd25fab-112b-4431-907e-7fb576d9af45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:46.730356 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd25fab-112b-4431-907e-7fb576d9af45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:46.731090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd25fab-112b-4431-907e-7fb576d9af45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:46.733895 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd25fab-112b-4431-907e-7fb576d9af45 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:46.749885 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 834/1676] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:28:46.750593 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-974311b1-26c4-4cdb-815d-b9305794a017 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be71c2333f684d91ac423548e450b5ee'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:46.750969 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd25fab-112b-4431-907e-7fb576d9af45 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:46.755271 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:46.764816 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-974311b1-26c4-4cdb-815d-b9305794a017 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be71c2333f684d91ac423548e450b5ee'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:46.767275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd25fab-112b-4431-907e-7fb576d9af45 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:46.775553 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 843/1677] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:46 2022] DELETE /identity/v3/users/be71c2333f684d91ac423548e450b5ee => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:46.776659 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3614b64-f2a2-4918-9e7e-bf5ababf69d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:46.776792 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974311b1-26c4-4cdb-815d-b9305794a017 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:46.777680 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3614b64-f2a2-4918-9e7e-bf5ababf69d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:46.778173 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3614b64-f2a2-4918-9e7e-bf5ababf69d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:46.778274 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-974311b1-26c4-4cdb-815d-b9305794a017 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:46.779208 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3614b64-f2a2-4918-9e7e-bf5ababf69d7 admin admin] PATH_INFO: `/v3/projects/5b4630939c7d48a583db0333e9af63e8/users/a68d9b0645194bca818ffab852b1ecef/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:46.795462 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:46.804670 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3614b64-f2a2-4918-9e7e-bf5ababf69d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b4630939c7d48a583db0333e9af63e8, user_id=a68d9b0645194bca818ffab852b1ecef, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:46.808788 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3614b64-f2a2-4918-9e7e-bf5ababf69d7 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:46.830863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 835/1678] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:46 2022] PUT /identity/v3/projects/5b4630939c7d48a583db0333e9af63e8/users/a68d9b0645194bca818ffab852b1ecef/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:46.834065 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9cdf1e-8714-4a88-8753-43dae6f6b544 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:46.834585 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9cdf1e-8714-4a88-8753-43dae6f6b544 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:46.835185 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9cdf1e-8714-4a88-8753-43dae6f6b544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:46.856658 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-5c9cdf1e-8714-4a88-8753-43dae6f6b544 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:46.857887 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-5c9cdf1e-8714-4a88-8753-43dae6f6b544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:46.864163 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974311b1-26c4-4cdb-815d-b9305794a017 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:46.924375 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 836/1679] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:46.926312 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce84dd05-48d6-42da-9827-a61285e82d7a None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:46.926602 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce84dd05-48d6-42da-9827-a61285e82d7a None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:46.926888 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce84dd05-48d6-42da-9827-a61285e82d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:46.940425 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b424a-a5c6-4e38-b6b1-4362c6c9a791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:46.941088 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b424a-a5c6-4e38-b6b1-4362c6c9a791 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:46.941367 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b424a-a5c6-4e38-b6b1-4362c6c9a791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:46.941670 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b424a-a5c6-4e38-b6b1-4362c6c9a791 admin admin] PATH_INFO: `/v3/projects/0ba5cab8660f4702b4f40b19dd5f809d` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:46.942973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7b424a-a5c6-4e38-b6b1-4362c6c9a791 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ba5cab8660f4702b4f40b19dd5f809d)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:46.945209 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7b424a-a5c6-4e38-b6b1-4362c6c9a791 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:46.963033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-ce84dd05-48d6-42da-9827-a61285e82d7a None None] MFA Rules not processed for user `a68d9b0645194bca818ffab852b1ecef`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:46.964294 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-ce84dd05-48d6-42da-9827-a61285e82d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:46.999078 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-cb7b424a-a5c6-4e38-b6b1-4362c6c9a791 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ba5cab8660f4702b4f40b19dd5f809d', 'request_id': 'req-cb7b424a-a5c6-4e38-b6b1-4362c6c9a791'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:47.010916 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-cb7b424a-a5c6-4e38-b6b1-4362c6c9a791 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ba5cab8660f4702b4f40b19dd5f809d', 'request_id': 'req-cb7b424a-a5c6-4e38-b6b1-4362c6c9a791'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:47.017441 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-cb7b424a-a5c6-4e38-b6b1-4362c6c9a791 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ba5cab8660f4702b4f40b19dd5f809d', 'request_id': 'req-cb7b424a-a5c6-4e38-b6b1-4362c6c9a791'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:47.030032 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-cb7b424a-a5c6-4e38-b6b1-4362c6c9a791 admin admin] The token cache is being invalidate because project 0ba5cab8660f4702b4f40b19dd5f809d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:47.030321 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-cb7b424a-a5c6-4e38-b6b1-4362c6c9a791 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:47.031069 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-cb7b424a-a5c6-4e38-b6b1-4362c6c9a791 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:47.032590 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 844/1680] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:46 2022] DELETE /identity/v3/projects/0ba5cab8660f4702b4f40b19dd5f809d => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:47.033491 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b424a-a5c6-4e38-b6b1-4362c6c9a791 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:47.034767 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-cb7b424a-a5c6-4e38-b6b1-4362c6c9a791 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:47.046550 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 837/1681] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:47.047445 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce84dd05-48d6-42da-9827-a61285e82d7a None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:47.058191 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce84dd05-48d6-42da-9827-a61285e82d7a None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:47.071721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:47.101406 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b424a-a5c6-4e38-b6b1-4362c6c9a791 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:47.106438 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369073f6-b0a0-4425-8b84-6e2b8a8f907d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:47.107712 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369073f6-b0a0-4425-8b84-6e2b8a8f907d service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:47.108209 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369073f6-b0a0-4425-8b84-6e2b8a8f907d service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:47.108695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369073f6-b0a0-4425-8b84-6e2b8a8f907d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:47.110734 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-369073f6-b0a0-4425-8b84-6e2b8a8f907d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:47.122504 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3560ad24-be9b-4dfa-8ad7-c7b535b8bb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:47.123175 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3560ad24-be9b-4dfa-8ad7-c7b535b8bb94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:47.123461 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3560ad24-be9b-4dfa-8ad7-c7b535b8bb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:47.123732 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3560ad24-be9b-4dfa-8ad7-c7b535b8bb94 admin admin] PATH_INFO: `/v3/users/56d64cd6d7d44dc089cf25c00949504b` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:47.124868 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3560ad24-be9b-4dfa-8ad7-c7b535b8bb94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56d64cd6d7d44dc089cf25c00949504b)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:47.127213 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3560ad24-be9b-4dfa-8ad7-c7b535b8bb94 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:47.157505 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:47.186902 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-3560ad24-be9b-4dfa-8ad7-c7b535b8bb94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56d64cd6d7d44dc089cf25c00949504b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:47.187543 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-3560ad24-be9b-4dfa-8ad7-c7b535b8bb94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56d64cd6d7d44dc089cf25c00949504b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:47.193224 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369073f6-b0a0-4425-8b84-6e2b8a8f907d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:47.194698 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369073f6-b0a0-4425-8b84-6e2b8a8f907d service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:47.194804 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-3560ad24-be9b-4dfa-8ad7-c7b535b8bb94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56d64cd6d7d44dc089cf25c00949504b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:47.204509 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 845/1682] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:47.205129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369073f6-b0a0-4425-8b84-6e2b8a8f907d service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:47.206545 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-3560ad24-be9b-4dfa-8ad7-c7b535b8bb94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56d64cd6d7d44dc089cf25c00949504b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:47.206736 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-369073f6-b0a0-4425-8b84-6e2b8a8f907d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:47.216893 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-3560ad24-be9b-4dfa-8ad7-c7b535b8bb94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56d64cd6d7d44dc089cf25c00949504b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:47.218954 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369073f6-b0a0-4425-8b84-6e2b8a8f907d service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:47.224696 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b681dc-72c0-4573-827c-28d18615d825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:47.224888 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 838/1683] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:47 2022] DELETE /identity/v3/users/56d64cd6d7d44dc089cf25c00949504b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:47.225262 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b681dc-72c0-4573-827c-28d18615d825 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:47.225544 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b681dc-72c0-4573-827c-28d18615d825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:47.225816 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b681dc-72c0-4573-827c-28d18615d825 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:47.226554 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b681dc-72c0-4573-827c-28d18615d825 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:47.228558 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b681dc-72c0-4573-827c-28d18615d825 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:47.258802 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 846/1684] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:47 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:47.263615 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3560ad24-be9b-4dfa-8ad7-c7b535b8bb94 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:47.265752 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-3560ad24-be9b-4dfa-8ad7-c7b535b8bb94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:47.468972 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3560ad24-be9b-4dfa-8ad7-c7b535b8bb94 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:47.615939 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b681dc-72c0-4573-827c-28d18615d825 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:47.617309 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90947ccf-ed6e-462e-88d1-f91bb9c5b562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:47.617906 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-26b681dc-72c0-4573-827c-28d18615d825 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:47.618160 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90947ccf-ed6e-462e-88d1-f91bb9c5b562 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:47.618521 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90947ccf-ed6e-462e-88d1-f91bb9c5b562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:47.618805 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90947ccf-ed6e-462e-88d1-f91bb9c5b562 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:47.619958 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90947ccf-ed6e-462e-88d1-f91bb9c5b562 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:47.622599 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90947ccf-ed6e-462e-88d1-f91bb9c5b562 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:47.651077 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b681dc-72c0-4573-827c-28d18615d825 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:47.664395 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78489b05-fe96-4af6-8daf-960356b87ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:47.666328 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78489b05-fe96-4af6-8daf-960356b87ea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:47.667097 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78489b05-fe96-4af6-8daf-960356b87ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:47.667375 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78489b05-fe96-4af6-8daf-960356b87ea9 admin admin] PATH_INFO: `/v3/projects/042319c18f6c49fbb748edfe946d1399` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:47.676637 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78489b05-fe96-4af6-8daf-960356b87ea9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=042319c18f6c49fbb748edfe946d1399)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:47.678772 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 839/1685] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:47 2022] POST /identity/v3/users => generated 297 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:47.682334 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90947ccf-ed6e-462e-88d1-f91bb9c5b562 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:47.683321 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78489b05-fe96-4af6-8daf-960356b87ea9 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:47.706904 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90947ccf-ed6e-462e-88d1-f91bb9c5b562 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:47.733711 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490b762-0a13-45ec-a787-c31fd290d73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:47.739050 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490b762-0a13-45ec-a787-c31fd290d73e admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:47.739620 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490b762-0a13-45ec-a787-c31fd290d73e admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:47.740174 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490b762-0a13-45ec-a787-c31fd290d73e admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:47.742318 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9490b762-0a13-45ec-a787-c31fd290d73e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:47.745292 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9490b762-0a13-45ec-a787-c31fd290d73e admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:47.774220 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 840/1686] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:47.777943 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490b762-0a13-45ec-a787-c31fd290d73e admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:47.806574 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490b762-0a13-45ec-a787-c31fd290d73e admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:47.820755 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-78489b05-fe96-4af6-8daf-960356b87ea9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '042319c18f6c49fbb748edfe946d1399', 'request_id': 'req-78489b05-fe96-4af6-8daf-960356b87ea9'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:47.843655 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-78489b05-fe96-4af6-8daf-960356b87ea9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '042319c18f6c49fbb748edfe946d1399', 'request_id': 'req-78489b05-fe96-4af6-8daf-960356b87ea9'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:47.858401 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-78489b05-fe96-4af6-8daf-960356b87ea9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '042319c18f6c49fbb748edfe946d1399', 'request_id': 'req-78489b05-fe96-4af6-8daf-960356b87ea9'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:47.865837 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:47.886943 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-78489b05-fe96-4af6-8daf-960356b87ea9 admin admin] The token cache is being invalidate because project 042319c18f6c49fbb748edfe946d1399 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:47.887492 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-78489b05-fe96-4af6-8daf-960356b87ea9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:47.888813 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-78489b05-fe96-4af6-8daf-960356b87ea9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:47.890542 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 847/1687] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:47 2022] DELETE /identity/v3/projects/042319c18f6c49fbb748edfe946d1399 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:47.891879 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78489b05-fe96-4af6-8daf-960356b87ea9 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:47.898945 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-78489b05-fe96-4af6-8daf-960356b87ea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:47.942756 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c63b27-de27-46f2-8d84-8b0fd28ba962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:47.942756 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c63b27-de27-46f2-8d84-8b0fd28ba962 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:47.942756 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c63b27-de27-46f2-8d84-8b0fd28ba962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:47.942756 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c63b27-de27-46f2-8d84-8b0fd28ba962 admin admin] PATH_INFO: `/v3/projects/90e272e295fb4aa7af0d4d3e76ccad79/users/f6fc7a97119b4c44b20bf98ea9fa117e/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:47.984851 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c63b27-de27-46f2-8d84-8b0fd28ba962 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90e272e295fb4aa7af0d4d3e76ccad79, user_id=f6fc7a97119b4c44b20bf98ea9fa117e, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:47.989816 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c63b27-de27-46f2-8d84-8b0fd28ba962 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:48.011906 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:48.021494 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 841/1688] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:47 2022] PUT /identity/v3/projects/90e272e295fb4aa7af0d4d3e76ccad79/users/f6fc7a97119b4c44b20bf98ea9fa117e/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:48.034080 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242f22e-f322-45a5-951d-6ff6e7376206 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:48.034365 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242f22e-f322-45a5-951d-6ff6e7376206 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:48.034631 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242f22e-f322-45a5-951d-6ff6e7376206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:48.051013 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78489b05-fe96-4af6-8daf-960356b87ea9 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:48.064559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-0242f22e-f322-45a5-951d-6ff6e7376206 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:48.066009 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-0242f22e-f322-45a5-951d-6ff6e7376206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:48.162640 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8880c60b-28bb-4f3f-942b-a0535e5669f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:48.163493 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880c60b-28bb-4f3f-942b-a0535e5669f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:48.164090 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880c60b-28bb-4f3f-942b-a0535e5669f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:48.164511 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880c60b-28bb-4f3f-942b-a0535e5669f6 admin admin] PATH_INFO: `/v3/projects/879377024bde46f38464aebc617e9c69` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:48.170244 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8880c60b-28bb-4f3f-942b-a0535e5669f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=879377024bde46f38464aebc617e9c69)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:48.186968 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 842/1689] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:48.188849 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8880c60b-28bb-4f3f-942b-a0535e5669f6 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:48.189693 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242f22e-f322-45a5-951d-6ff6e7376206 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:48.200935 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-0242f22e-f322-45a5-951d-6ff6e7376206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:48.220639 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242f22e-f322-45a5-951d-6ff6e7376206 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:48.234992 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aadadc3-c90a-43f8-b190-e387cf9ffb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:48.235719 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aadadc3-c90a-43f8-b190-e387cf9ffb56 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:48.236011 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aadadc3-c90a-43f8-b190-e387cf9ffb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:48.236263 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aadadc3-c90a-43f8-b190-e387cf9ffb56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:48.236964 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aadadc3-c90a-43f8-b190-e387cf9ffb56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:48.238699 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aadadc3-c90a-43f8-b190-e387cf9ffb56 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:48.265214 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 843/1690] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:48.267569 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dafd3-ba09-4018-8226-f45312c66fc2 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:48.268491 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dafd3-ba09-4018-8226-f45312c66fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:48.269215 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dafd3-ba09-4018-8226-f45312c66fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:48.294224 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-645dafd3-ba09-4018-8226-f45312c66fc2 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:48.296151 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-645dafd3-ba09-4018-8226-f45312c66fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:48.317502 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-8880c60b-28bb-4f3f-942b-a0535e5669f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '879377024bde46f38464aebc617e9c69', 'request_id': 'req-8880c60b-28bb-4f3f-942b-a0535e5669f6'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:48.336803 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-8880c60b-28bb-4f3f-942b-a0535e5669f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '879377024bde46f38464aebc617e9c69', 'request_id': 'req-8880c60b-28bb-4f3f-942b-a0535e5669f6'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:48.349582 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-8880c60b-28bb-4f3f-942b-a0535e5669f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '879377024bde46f38464aebc617e9c69', 'request_id': 'req-8880c60b-28bb-4f3f-942b-a0535e5669f6'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:48.364896 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-8880c60b-28bb-4f3f-942b-a0535e5669f6 admin admin] The token cache is being invalidate because project 879377024bde46f38464aebc617e9c69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:48.365196 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-8880c60b-28bb-4f3f-942b-a0535e5669f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:48.365840 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-8880c60b-28bb-4f3f-942b-a0535e5669f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:48.371667 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 848/1691] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:47 2022] DELETE /identity/v3/projects/879377024bde46f38464aebc617e9c69 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:48.371667 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8880c60b-28bb-4f3f-942b-a0535e5669f6 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:48.378931 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-8880c60b-28bb-4f3f-942b-a0535e5669f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:48.404894 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 844/1692] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:48.406803 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a7a16-343e-4527-b51d-dd2979a25b5f None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:48.407243 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a7a16-343e-4527-b51d-dd2979a25b5f None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:48.407511 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a7a16-343e-4527-b51d-dd2979a25b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:48.428976 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-d27a7a16-343e-4527-b51d-dd2979a25b5f None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:48.430502 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-d27a7a16-343e-4527-b51d-dd2979a25b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:48.439289 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 845/1693] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:48.446922 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a7a16-343e-4527-b51d-dd2979a25b5f None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:48.463377 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a7a16-343e-4527-b51d-dd2979a25b5f None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:48.469982 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133a6f1-56cd-4ab6-8d88-1312ba7a9653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:48.471219 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133a6f1-56cd-4ab6-8d88-1312ba7a9653 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:48.471609 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133a6f1-56cd-4ab6-8d88-1312ba7a9653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:48.472147 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133a6f1-56cd-4ab6-8d88-1312ba7a9653 admin admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:48.473033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133a6f1-56cd-4ab6-8d88-1312ba7a9653 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:48.473538 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133a6f1-56cd-4ab6-8d88-1312ba7a9653 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:48.475686 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133a6f1-56cd-4ab6-8d88-1312ba7a9653 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:48.489543 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 846/1694] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:28:48.490862 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f3d3a0-08a3-438f-9047-97c13842473b None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:48.491236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f3d3a0-08a3-438f-9047-97c13842473b None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:48.491578 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f3d3a0-08a3-438f-9047-97c13842473b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:48.509166 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8880c60b-28bb-4f3f-942b-a0535e5669f6 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:48.511980 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-39f3d3a0-08a3-438f-9047-97c13842473b None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:48.513190 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-39f3d3a0-08a3-438f-9047-97c13842473b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:48.514187 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab6742-4b38-44c6-be2a-bf26ab270c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:48.515077 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab6742-4b38-44c6-be2a-bf26ab270c8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:48.515444 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab6742-4b38-44c6-be2a-bf26ab270c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:48.515796 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab6742-4b38-44c6-be2a-bf26ab270c8b admin admin] PATH_INFO: `/v3/projects/90e272e295fb4aa7af0d4d3e76ccad79/users/f6fc7a97119b4c44b20bf98ea9fa117e/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:48.517755 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eab6742-4b38-44c6-be2a-bf26ab270c8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=90e272e295fb4aa7af0d4d3e76ccad79, user_id=f6fc7a97119b4c44b20bf98ea9fa117e, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:48.520709 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eab6742-4b38-44c6-be2a-bf26ab270c8b admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:48.525301 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 847/1695] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:48.525301 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f3d3a0-08a3-438f-9047-97c13842473b None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:48.534839 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f3d3a0-08a3-438f-9047-97c13842473b None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:48.541987 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 849/1696] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:48 2022] PUT /identity/v3/projects/90e272e295fb4aa7af0d4d3e76ccad79/users/f6fc7a97119b4c44b20bf98ea9fa117e/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:48.541987 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab6742-4b38-44c6-be2a-bf26ab270c8b admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:48.550324 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:48.563498 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:48.604366 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab3dd1-8840-498a-a6d8-0217376ea26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:48.604985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab3dd1-8840-498a-a6d8-0217376ea26f admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:48.605254 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab3dd1-8840-498a-a6d8-0217376ea26f admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:48.605801 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab3dd1-8840-498a-a6d8-0217376ea26f admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:48.606435 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab3dd1-8840-498a-a6d8-0217376ea26f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:48.608406 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab3dd1-8840-498a-a6d8-0217376ea26f admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:48.613135 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab6742-4b38-44c6-be2a-bf26ab270c8b admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:48.617859 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3c0b6-1527-45a4-95af-392bc50b38a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:48.618405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3c0b6-1527-45a4-95af-392bc50b38a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:48.618747 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3c0b6-1527-45a4-95af-392bc50b38a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:48.619023 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3c0b6-1527-45a4-95af-392bc50b38a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:48.619692 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3c0b6-1527-45a4-95af-392bc50b38a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:48.619994 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3c0b6-1527-45a4-95af-392bc50b38a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:48.621592 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3c0b6-1527-45a4-95af-392bc50b38a7 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:48.634376 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 850/1697] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:28:48.637044 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 848/1698] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:48 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:48.641000 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab3dd1-8840-498a-a6d8-0217376ea26f admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:48.641568 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170c6e3-9368-4e43-ad62-af14381c1b12 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:48.641850 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170c6e3-9368-4e43-ad62-af14381c1b12 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:48.642122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170c6e3-9368-4e43-ad62-af14381c1b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:48.669521 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-a170c6e3-9368-4e43-ad62-af14381c1b12 None None] MFA Rules not processed for user `f6fc7a97119b4c44b20bf98ea9fa117e`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:48.670775 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-a170c6e3-9368-4e43-ad62-af14381c1b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:48.701534 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab3dd1-8840-498a-a6d8-0217376ea26f admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:48.709219 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115b4422-ede7-48da-a9b1-e15cf447e77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:48.709855 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b4422-ede7-48da-a9b1-e15cf447e77a admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:48.710109 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b4422-ede7-48da-a9b1-e15cf447e77a admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:48.710372 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b4422-ede7-48da-a9b1-e15cf447e77a admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:48.711033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115b4422-ede7-48da-a9b1-e15cf447e77a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:48.713587 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115b4422-ede7-48da-a9b1-e15cf447e77a admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:48.738612 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 849/1699] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:48 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:28:48.742434 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115b4422-ede7-48da-a9b1-e15cf447e77a admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:48.743245 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 851/1700] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:48 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:48.744381 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170c6e3-9368-4e43-ad62-af14381c1b12 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:48.757379 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115b4422-ede7-48da-a9b1-e15cf447e77a admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:48.761698 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170c6e3-9368-4e43-ad62-af14381c1b12 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:48.762050 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37fee76-ab6d-4e24-90fe-ebdb936a7d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:48.762698 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37fee76-ab6d-4e24-90fe-ebdb936a7d78 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:48.763283 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37fee76-ab6d-4e24-90fe-ebdb936a7d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:48.763651 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37fee76-ab6d-4e24-90fe-ebdb936a7d78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:48.765670 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37fee76-ab6d-4e24-90fe-ebdb936a7d78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:48.766269 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37fee76-ab6d-4e24-90fe-ebdb936a7d78 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:48.768076 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b5f77e-060a-4795-8fab-b090c5e657aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:48.768767 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5f77e-060a-4795-8fab-b090c5e657aa admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:48.769107 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5f77e-060a-4795-8fab-b090c5e657aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:48.769448 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5f77e-060a-4795-8fab-b090c5e657aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:48.770276 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b5f77e-060a-4795-8fab-b090c5e657aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:48.772275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b5f77e-060a-4795-8fab-b090c5e657aa admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:48.786957 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 852/1701] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:48.787949 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b5f77e-060a-4795-8fab-b090c5e657aa admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:48.790032 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-d4b5f77e-060a-4795-8fab-b090c5e657aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:48.796224 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 850/1702] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:48 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:48.797339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37fee76-ab6d-4e24-90fe-ebdb936a7d78 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:48.861917 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b5f77e-060a-4795-8fab-b090c5e657aa admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:48.864476 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37fee76-ab6d-4e24-90fe-ebdb936a7d78 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:48.869899 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40411606-a3d2-4754-ab50-6422ae9a4835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:48.870828 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40411606-a3d2-4754-ab50-6422ae9a4835 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:48.871175 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40411606-a3d2-4754-ab50-6422ae9a4835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:48.871506 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40411606-a3d2-4754-ab50-6422ae9a4835 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:48.872222 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40411606-a3d2-4754-ab50-6422ae9a4835 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:48.874764 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40411606-a3d2-4754-ab50-6422ae9a4835 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:48.881900 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:48.891969 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 851/1703] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:48.893186 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40411606-a3d2-4754-ab50-6422ae9a4835 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:48.909465 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:48.915706 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40411606-a3d2-4754-ab50-6422ae9a4835 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:48.921451 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8970b68e-5b4a-4ae5-a072-053adf309100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:48.921793 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43191c8-8a1c-4eb1-91ba-611768984028 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:48.922348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8970b68e-5b4a-4ae5-a072-053adf309100 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:48.922579 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43191c8-8a1c-4eb1-91ba-611768984028 service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:48.922779 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8970b68e-5b4a-4ae5-a072-053adf309100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:48.923042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43191c8-8a1c-4eb1-91ba-611768984028 service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:48.923220 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8970b68e-5b4a-4ae5-a072-053adf309100 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:48.923416 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43191c8-8a1c-4eb1-91ba-611768984028 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:48.924012 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8970b68e-5b4a-4ae5-a072-053adf309100 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:48.925979 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8970b68e-5b4a-4ae5-a072-053adf309100 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:48.955813 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 852/1704] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:48 2022] POST /identity/v3/users => generated 292 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:48.960948 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8970b68e-5b4a-4ae5-a072-053adf309100 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:48.979538 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8970b68e-5b4a-4ae5-a072-053adf309100 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:48.986177 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfbb7af-31d0-40f8-b4cb-8e8cf677e9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:48.986886 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbb7af-31d0-40f8-b4cb-8e8cf677e9a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:48.987327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbb7af-31d0-40f8-b4cb-8e8cf677e9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:48.987605 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbb7af-31d0-40f8-b4cb-8e8cf677e9a3 admin admin] PATH_INFO: `/v3/projects/6df0bcee2fc949c789b67b2e64b06cd0/users/e232cf99f83a4501afcbc07686f10904/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:48.992498 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43191c8-8a1c-4eb1-91ba-611768984028 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:48.994310 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43191c8-8a1c-4eb1-91ba-611768984028 service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:49.005160 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfbb7af-31d0-40f8-b4cb-8e8cf677e9a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6df0bcee2fc949c789b67b2e64b06cd0, user_id=e232cf99f83a4501afcbc07686f10904, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:49.008608 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 853/1705] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:48 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:49.009738 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfbb7af-31d0-40f8-b4cb-8e8cf677e9a3 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:49.009882 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43191c8-8a1c-4eb1-91ba-611768984028 service cinder] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:49.023907 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43191c8-8a1c-4eb1-91ba-611768984028 service cinder] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:49.030779 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 853/1706] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:48 2022] PUT /identity/v3/projects/6df0bcee2fc949c789b67b2e64b06cd0/users/e232cf99f83a4501afcbc07686f10904/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:49.041538 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6d960-c75e-4f83-a148-a46300bfdd43 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:49.041718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6d960-c75e-4f83-a148-a46300bfdd43 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:49.041987 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6d960-c75e-4f83-a148-a46300bfdd43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:49.100061 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-4ed6d960-c75e-4f83-a148-a46300bfdd43 None None] MFA Rules not processed for user `e232cf99f83a4501afcbc07686f10904`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:49.101562 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-4ed6d960-c75e-4f83-a148-a46300bfdd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:49.112671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5a3ee-e261-4006-af3d-5a85eb92f4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:49.114180 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5a3ee-e261-4006-af3d-5a85eb92f4e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:49.114180 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5a3ee-e261-4006-af3d-5a85eb92f4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:49.115092 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5a3ee-e261-4006-af3d-5a85eb92f4e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:49.116339 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5a3ee-e261-4006-af3d-5a85eb92f4e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:49.119810 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5a3ee-e261-4006-af3d-5a85eb92f4e1 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:49.141862 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 854/1707] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:49.143789 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5a3ee-e261-4006-af3d-5a85eb92f4e1 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:49.161688 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5a3ee-e261-4006-af3d-5a85eb92f4e1 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:49.168296 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59120955-09b5-44bc-a8f8-1928da8beb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:49.169590 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59120955-09b5-44bc-a8f8-1928da8beb3e admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:49.169873 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59120955-09b5-44bc-a8f8-1928da8beb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:49.170239 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59120955-09b5-44bc-a8f8-1928da8beb3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:49.170988 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59120955-09b5-44bc-a8f8-1928da8beb3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:49.173116 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59120955-09b5-44bc-a8f8-1928da8beb3e admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:49.185584 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 855/1708] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:49.189944 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59120955-09b5-44bc-a8f8-1928da8beb3e admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:49.204835 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59120955-09b5-44bc-a8f8-1928da8beb3e admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:49.210215 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32708a3d-90cb-4620-bd47-944681adf871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:49.211729 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32708a3d-90cb-4620-bd47-944681adf871 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:49.211984 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32708a3d-90cb-4620-bd47-944681adf871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:49.213168 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32708a3d-90cb-4620-bd47-944681adf871 admin admin] PATH_INFO: `/v3/projects/21c827bae43347d4b3537d5036df9548/users/1bf9cc586c5b4c9cb501939c587d6a6e/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:49.234005 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 854/1709] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:49.250392 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32708a3d-90cb-4620-bd47-944681adf871 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21c827bae43347d4b3537d5036df9548, user_id=1bf9cc586c5b4c9cb501939c587d6a6e, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:49.253779 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32708a3d-90cb-4620-bd47-944681adf871 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:49.263666 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6d960-c75e-4f83-a148-a46300bfdd43 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:49.266032 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-4ed6d960-c75e-4f83-a148-a46300bfdd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:49.274971 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 856/1710] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:49 2022] PUT /identity/v3/projects/21c827bae43347d4b3537d5036df9548/users/1bf9cc586c5b4c9cb501939c587d6a6e/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:49.286132 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd1141-bbcc-4001-9a81-4d12f0e46d3d None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:49.286406 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd1141-bbcc-4001-9a81-4d12f0e46d3d None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:49.287154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd1141-bbcc-4001-9a81-4d12f0e46d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:49.340650 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-d2cd1141-bbcc-4001-9a81-4d12f0e46d3d None None] MFA Rules not processed for user `1bf9cc586c5b4c9cb501939c587d6a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:49.342494 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-d2cd1141-bbcc-4001-9a81-4d12f0e46d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:49.401724 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6d960-c75e-4f83-a148-a46300bfdd43 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:49.417871 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:49.473088 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 857/1711] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:49.477607 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd1141-bbcc-4001-9a81-4d12f0e46d3d None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:49.508879 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde0faa1-7e0d-4c86-9dd9-6ce4bb4670f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:49.509759 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde0faa1-7e0d-4c86-9dd9-6ce4bb4670f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:49.510094 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde0faa1-7e0d-4c86-9dd9-6ce4bb4670f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:49.510409 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde0faa1-7e0d-4c86-9dd9-6ce4bb4670f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:49.538464 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:49.615534 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd1141-bbcc-4001-9a81-4d12f0e46d3d None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:49.626476 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c249e2ea-ea76-4363-bd13-224ef76656ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:49.628793 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249e2ea-ea76-4363-bd13-224ef76656ed admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:49.629393 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249e2ea-ea76-4363-bd13-224ef76656ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:49.629965 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249e2ea-ea76-4363-bd13-224ef76656ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:49.631558 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c249e2ea-ea76-4363-bd13-224ef76656ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:49.635598 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c249e2ea-ea76-4363-bd13-224ef76656ed admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:49.647639 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde0faa1-7e0d-4c86-9dd9-6ce4bb4670f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:49.651454 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde0faa1-7e0d-4c86-9dd9-6ce4bb4670f0 service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:49.669639 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 855/1712] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:28:49.698438 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 858/1713] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:49 2022] POST /identity/v3/projects => generated 383 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:49.704269 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde0faa1-7e0d-4c86-9dd9-6ce4bb4670f0 service cinder] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:49.724982 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde0faa1-7e0d-4c86-9dd9-6ce4bb4670f0 service cinder] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:49.803600 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c249e2ea-ea76-4363-bd13-224ef76656ed admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:49.816875 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3095be-4e43-4871-93f7-a2146c26e8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:49.817847 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3095be-4e43-4871-93f7-a2146c26e8ee admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:49.818316 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3095be-4e43-4871-93f7-a2146c26e8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:49.818820 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3095be-4e43-4871-93f7-a2146c26e8ee admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:49.819614 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3095be-4e43-4871-93f7-a2146c26e8ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:49.822399 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3095be-4e43-4871-93f7-a2146c26e8ee admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:49.869041 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 856/1714] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:49 2022] POST /identity/v3/users => generated 298 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:49.870463 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3095be-4e43-4871-93f7-a2146c26e8ee admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:49.885139 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3095be-4e43-4871-93f7-a2146c26e8ee admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:49.892222 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4694d6-acb0-47fc-81aa-ced3ee4fd9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:49.892441 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4694d6-acb0-47fc-81aa-ced3ee4fd9f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:49.893807 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4694d6-acb0-47fc-81aa-ced3ee4fd9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:49.893807 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4694d6-acb0-47fc-81aa-ced3ee4fd9f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:49.895342 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4694d6-acb0-47fc-81aa-ced3ee4fd9f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:49.897517 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4694d6-acb0-47fc-81aa-ced3ee4fd9f4 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:49.904836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c249e2ea-ea76-4363-bd13-224ef76656ed admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:49.914456 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd95baf-b2ae-4d19-bf17-d679f3bea8f4 tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6fc7a97119b4c44b20bf98ea9fa117e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90e272e295fb4aa7af0d4d3e76ccad79', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:49.915675 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd95baf-b2ae-4d19-bf17-d679f3bea8f4 tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:49.916609 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd95baf-b2ae-4d19-bf17-d679f3bea8f4 tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:49.917314 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd95baf-b2ae-4d19-bf17-d679f3bea8f4 tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:49.918250 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd95baf-b2ae-4d19-bf17-d679f3bea8f4 tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:49.918754 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd95baf-b2ae-4d19-bf17-d679f3bea8f4 tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:49.921150 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd95baf-b2ae-4d19-bf17-d679f3bea8f4 tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:49.922777 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 857/1715] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:49.927909 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4694d6-acb0-47fc-81aa-ced3ee4fd9f4 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:49.945761 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4694d6-acb0-47fc-81aa-ced3ee4fd9f4 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:49.947948 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 859/1716] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:28:49.952437 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd95baf-b2ae-4d19-bf17-d679f3bea8f4 tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:49.965831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef57ac-abd7-4470-af06-59daeb54d606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:49.969040 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef57ac-abd7-4470-af06-59daeb54d606 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:49.969752 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef57ac-abd7-4470-af06-59daeb54d606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:49.971000 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef57ac-abd7-4470-af06-59daeb54d606 admin admin] PATH_INFO: `/v3/projects/93eb23200d80433ebfb35271045637b0/users/bd73809ca24242d4afbe8d963779bc9a/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:49.978382 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd95baf-b2ae-4d19-bf17-d679f3bea8f4 tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:49.984771 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f047062c-4dde-47c9-8c97-ef4dbcf10952 tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6fc7a97119b4c44b20bf98ea9fa117e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90e272e295fb4aa7af0d4d3e76ccad79', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:49.985614 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f047062c-4dde-47c9-8c97-ef4dbcf10952 tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:49.985991 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f047062c-4dde-47c9-8c97-ef4dbcf10952 tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:49.986693 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f047062c-4dde-47c9-8c97-ef4dbcf10952 tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:49.987486 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f047062c-4dde-47c9-8c97-ef4dbcf10952 tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:49.989526 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f047062c-4dde-47c9-8c97-ef4dbcf10952 tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:50.013637 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef57ac-abd7-4470-af06-59daeb54d606 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93eb23200d80433ebfb35271045637b0, user_id=bd73809ca24242d4afbe8d963779bc9a, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:50.019124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef57ac-abd7-4470-af06-59daeb54d606 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:50.045052 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 860/1717] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:49 2022] POST /identity/v3/projects => generated 354 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:50.049043 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f047062c-4dde-47c9-8c97-ef4dbcf10952 tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:50.097160 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 858/1718] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:49 2022] PUT /identity/v3/projects/93eb23200d80433ebfb35271045637b0/users/bd73809ca24242d4afbe8d963779bc9a/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:50.101250 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef57ac-abd7-4470-af06-59daeb54d606 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:50.145184 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:50.205243 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f047062c-4dde-47c9-8c97-ef4dbcf10952 tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:50.218975 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26627a-d58c-4d83-ae21-a069598c23d0 tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6fc7a97119b4c44b20bf98ea9fa117e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90e272e295fb4aa7af0d4d3e76ccad79', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:50.218975 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26627a-d58c-4d83-ae21-a069598c23d0 tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:50.218975 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26627a-d58c-4d83-ae21-a069598c23d0 tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:50.219365 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26627a-d58c-4d83-ae21-a069598c23d0 tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:50.219365 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee26627a-d58c-4d83-ae21-a069598c23d0 tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:50.219365 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee26627a-d58c-4d83-ae21-a069598c23d0 tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:50.220016 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee26627a-d58c-4d83-ae21-a069598c23d0 tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:50.243252 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 861/1719] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:28:50.280326 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef57ac-abd7-4470-af06-59daeb54d606 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:50.339442 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:50.434915 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a951b6a-6b8f-4cee-a377-8035a5ef61e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:50.435815 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a951b6a-6b8f-4cee-a377-8035a5ef61e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:50.436336 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a951b6a-6b8f-4cee-a377-8035a5ef61e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:50.436798 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a951b6a-6b8f-4cee-a377-8035a5ef61e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:50.438262 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a951b6a-6b8f-4cee-a377-8035a5ef61e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:50.439723 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26627a-d58c-4d83-ae21-a069598c23d0 tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:50.441892 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a951b6a-6b8f-4cee-a377-8035a5ef61e2 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:50.458472 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26627a-d58c-4d83-ae21-a069598c23d0 tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:50.466538 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 859/1720] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:50.473608 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a951b6a-6b8f-4cee-a377-8035a5ef61e2 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:50.504368 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64d084-cd9e-4ebb-9135-53487cb926fa tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6fc7a97119b4c44b20bf98ea9fa117e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90e272e295fb4aa7af0d4d3e76ccad79', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:50.504999 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64d084-cd9e-4ebb-9135-53487cb926fa tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:50.505286 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64d084-cd9e-4ebb-9135-53487cb926fa tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:50.505564 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64d084-cd9e-4ebb-9135-53487cb926fa tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] PATH_INFO: `/v3/projects/39142b7512fb412db69111776d3397ff` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:50.506053 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a951b6a-6b8f-4cee-a377-8035a5ef61e2 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:50.508313 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f64d084-cd9e-4ebb-9135-53487cb926fa tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] RBAC: Authorizing `identity:delete_project(project_id=39142b7512fb412db69111776d3397ff)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:50.512108 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf33db9-1ec7-4da3-b013-b2ce78a83cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:50.513144 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf33db9-1ec7-4da3-b013-b2ce78a83cfc admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:50.513659 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf33db9-1ec7-4da3-b013-b2ce78a83cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:50.514121 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf33db9-1ec7-4da3-b013-b2ce78a83cfc admin admin] PATH_INFO: `/v3/projects/93eb23200d80433ebfb35271045637b0/users/bd73809ca24242d4afbe8d963779bc9a/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:50.516013 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f64d084-cd9e-4ebb-9135-53487cb926fa tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:50.516618 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf33db9-1ec7-4da3-b013-b2ce78a83cfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=93eb23200d80433ebfb35271045637b0, user_id=bd73809ca24242d4afbe8d963779bc9a, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:50.520178 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf33db9-1ec7-4da3-b013-b2ce78a83cfc admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:50.557470 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 860/1721] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:50 2022] PUT /identity/v3/projects/93eb23200d80433ebfb35271045637b0/users/bd73809ca24242d4afbe8d963779bc9a/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:50.569704 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb9903-b692-42ee-9678-a654e14910f6 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:50.572179 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb9903-b692-42ee-9678-a654e14910f6 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:50.572891 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb9903-b692-42ee-9678-a654e14910f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:50.657180 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-1ecb9903-b692-42ee-9678-a654e14910f6 None None] MFA Rules not processed for user `bd73809ca24242d4afbe8d963779bc9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:50.658788 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-1ecb9903-b692-42ee-9678-a654e14910f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:50.662870 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-1f64d084-cd9e-4ebb-9135-53487cb926fa tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39142b7512fb412db69111776d3397ff', 'request_id': 'req-1f64d084-cd9e-4ebb-9135-53487cb926fa'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:50.674539 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-1f64d084-cd9e-4ebb-9135-53487cb926fa tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39142b7512fb412db69111776d3397ff', 'request_id': 'req-1f64d084-cd9e-4ebb-9135-53487cb926fa'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:50.683692 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-1f64d084-cd9e-4ebb-9135-53487cb926fa tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39142b7512fb412db69111776d3397ff', 'request_id': 'req-1f64d084-cd9e-4ebb-9135-53487cb926fa'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:50.701091 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-1f64d084-cd9e-4ebb-9135-53487cb926fa tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] The token cache is being invalidate because project 39142b7512fb412db69111776d3397ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:50.701389 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-1f64d084-cd9e-4ebb-9135-53487cb926fa tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:50.702108 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-1f64d084-cd9e-4ebb-9135-53487cb926fa tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:50.707340 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 862/1722] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:50 2022] DELETE /identity/v3/projects/39142b7512fb412db69111776d3397ff => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:50.821884 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 861/1723] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:50 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:50.922102 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64d084-cd9e-4ebb-9135-53487cb926fa tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:50.930013 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-1f64d084-cd9e-4ebb-9135-53487cb926fa tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:51.032144 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64d084-cd9e-4ebb-9135-53487cb926fa tempest-VolumeQuotaClassesTest-1448928879 tempest-VolumeQuotaClassesTest-1448928879-project-admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:51.037933 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1017e-642e-4e8a-a695-afc00cf98710 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:51.038851 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1017e-642e-4e8a-a695-afc00cf98710 service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:51.039262 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1017e-642e-4e8a-a695-afc00cf98710 service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:51.039724 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1017e-642e-4e8a-a695-afc00cf98710 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:51.042306 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-fed1017e-642e-4e8a-a695-afc00cf98710 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:51.074998 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb9903-b692-42ee-9678-a654e14910f6 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:51.076747 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-1ecb9903-b692-42ee-9678-a654e14910f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:51.129140 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed1017e-642e-4e8a-a695-afc00cf98710 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:51.130906 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed1017e-642e-4e8a-a695-afc00cf98710 service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:51.145947 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 863/1724] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:51.167992 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb9903-b692-42ee-9678-a654e14910f6 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:51.172683 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db75f1-fb97-442d-ab86-f3600d94bbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:51.173373 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db75f1-fb97-442d-ab86-f3600d94bbac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:51.173659 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db75f1-fb97-442d-ab86-f3600d94bbac admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:51.173928 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db75f1-fb97-442d-ab86-f3600d94bbac admin admin] PATH_INFO: `/v3/users/a68d9b0645194bca818ffab852b1ecef` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:51.175253 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32db75f1-fb97-442d-ab86-f3600d94bbac admin admin] RBAC: Authorizing `identity:delete_user(user_id=a68d9b0645194bca818ffab852b1ecef)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:51.177242 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32db75f1-fb97-442d-ab86-f3600d94bbac admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:51.221947 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:51.240981 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-32db75f1-fb97-442d-ab86-f3600d94bbac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a68d9b0645194bca818ffab852b1ecef'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:51.241684 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-32db75f1-fb97-442d-ab86-f3600d94bbac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a68d9b0645194bca818ffab852b1ecef'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:51.249360 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-32db75f1-fb97-442d-ab86-f3600d94bbac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a68d9b0645194bca818ffab852b1ecef'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:51.262568 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-32db75f1-fb97-442d-ab86-f3600d94bbac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a68d9b0645194bca818ffab852b1ecef'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:51.270763 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-32db75f1-fb97-442d-ab86-f3600d94bbac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a68d9b0645194bca818ffab852b1ecef'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:51.278437 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 862/1725] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:51 2022] DELETE /identity/v3/users/a68d9b0645194bca818ffab852b1ecef => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:51.282879 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f97251-a4c6-4c77-ad9b-1dfbe7a79aab None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:51.283214 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f97251-a4c6-4c77-ad9b-1dfbe7a79aab None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:51.283478 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f97251-a4c6-4c77-ad9b-1dfbe7a79aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:51.303883 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-10f97251-a4c6-4c77-ad9b-1dfbe7a79aab None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:51.305211 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-10f97251-a4c6-4c77-ad9b-1dfbe7a79aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:51.309698 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:51.386403 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 864/1726] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:51.393087 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db75f1-fb97-442d-ab86-f3600d94bbac admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:51.395835 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-32db75f1-fb97-442d-ab86-f3600d94bbac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:51.453005 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7ac97-8a44-47d5-a521-16f6f6f08a36 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:51.453372 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7ac97-8a44-47d5-a521-16f6f6f08a36 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:51.453841 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7ac97-8a44-47d5-a521-16f6f6f08a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:51.471220 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db75f1-fb97-442d-ab86-f3600d94bbac admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:51.475096 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-f8e7ac97-8a44-47d5-a521-16f6f6f08a36 None None] MFA Rules not processed for user `e232cf99f83a4501afcbc07686f10904`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:51.476602 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-f8e7ac97-8a44-47d5-a521-16f6f6f08a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:51.477414 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903ec7a-3774-436b-93f7-34240d670580 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:51.478393 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903ec7a-3774-436b-93f7-34240d670580 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:51.479289 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903ec7a-3774-436b-93f7-34240d670580 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:51.479718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903ec7a-3774-436b-93f7-34240d670580 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:51.531134 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903ec7a-3774-436b-93f7-34240d670580 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:51.533090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903ec7a-3774-436b-93f7-34240d670580 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:51.562212 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 865/1727] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:51.564388 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 863/1728] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:52.032446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e7ac97-8a44-47d5-a521-16f6f6f08a36 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:52.033969 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-f8e7ac97-8a44-47d5-a521-16f6f6f08a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:52.070418 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e7ac97-8a44-47d5-a521-16f6f6f08a36 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:52.076360 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a335ac30-ebb8-46a0-a93a-c72661ccd487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:52.077112 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335ac30-ebb8-46a0-a93a-c72661ccd487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:52.077386 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335ac30-ebb8-46a0-a93a-c72661ccd487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:52.077770 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335ac30-ebb8-46a0-a93a-c72661ccd487 admin admin] PATH_INFO: `/v3/projects/5b4630939c7d48a583db0333e9af63e8` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:52.079608 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a335ac30-ebb8-46a0-a93a-c72661ccd487 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b4630939c7d48a583db0333e9af63e8)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:52.083306 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a335ac30-ebb8-46a0-a93a-c72661ccd487 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:52.137153 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-a335ac30-ebb8-46a0-a93a-c72661ccd487 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b4630939c7d48a583db0333e9af63e8', 'request_id': 'req-a335ac30-ebb8-46a0-a93a-c72661ccd487'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:52.152788 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-a335ac30-ebb8-46a0-a93a-c72661ccd487 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b4630939c7d48a583db0333e9af63e8', 'request_id': 'req-a335ac30-ebb8-46a0-a93a-c72661ccd487'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:52.159651 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-a335ac30-ebb8-46a0-a93a-c72661ccd487 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b4630939c7d48a583db0333e9af63e8', 'request_id': 'req-a335ac30-ebb8-46a0-a93a-c72661ccd487'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:52.172887 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-a335ac30-ebb8-46a0-a93a-c72661ccd487 admin admin] The token cache is being invalidate because project 5b4630939c7d48a583db0333e9af63e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:52.173299 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-a335ac30-ebb8-46a0-a93a-c72661ccd487 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:52.174204 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-a335ac30-ebb8-46a0-a93a-c72661ccd487 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:52.176204 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 866/1729] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:52 2022] DELETE /identity/v3/projects/5b4630939c7d48a583db0333e9af63e8 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:52.179994 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903ec7a-3774-436b-93f7-34240d670580 service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:52.193230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903ec7a-3774-436b-93f7-34240d670580 service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:52.301757 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d12d36-4a4b-49a0-9357-a605ee5737b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:52.302373 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d12d36-4a4b-49a0-9357-a605ee5737b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:52.302604 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d12d36-4a4b-49a0-9357-a605ee5737b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:52.302915 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d12d36-4a4b-49a0-9357-a605ee5737b5 admin admin] PATH_INFO: `/v3/users/f6fc7a97119b4c44b20bf98ea9fa117e` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:52.304019 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d12d36-4a4b-49a0-9357-a605ee5737b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6fc7a97119b4c44b20bf98ea9fa117e)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:52.306529 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d12d36-4a4b-49a0-9357-a605ee5737b5 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:52.368587 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-f3d12d36-4a4b-49a0-9357-a605ee5737b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6fc7a97119b4c44b20bf98ea9fa117e'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:52.369476 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-f3d12d36-4a4b-49a0-9357-a605ee5737b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6fc7a97119b4c44b20bf98ea9fa117e'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:52.374544 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:52.380041 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-f3d12d36-4a4b-49a0-9357-a605ee5737b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6fc7a97119b4c44b20bf98ea9fa117e'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:52.397635 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-f3d12d36-4a4b-49a0-9357-a605ee5737b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6fc7a97119b4c44b20bf98ea9fa117e'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:52.418243 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-f3d12d36-4a4b-49a0-9357-a605ee5737b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6fc7a97119b4c44b20bf98ea9fa117e'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:52.424915 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 864/1730] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:52 2022] DELETE /identity/v3/users/f6fc7a97119b4c44b20bf98ea9fa117e => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:52.716165 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a335ac30-ebb8-46a0-a93a-c72661ccd487 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:52.717722 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-a335ac30-ebb8-46a0-a93a-c72661ccd487 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:52.731033 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a335ac30-ebb8-46a0-a93a-c72661ccd487 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:52.766559 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:52.791404 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda0823-9e9c-4423-8769-b2c5dfa96eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:52.792031 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda0823-9e9c-4423-8769-b2c5dfa96eef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:52.792305 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda0823-9e9c-4423-8769-b2c5dfa96eef admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:52.792563 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda0823-9e9c-4423-8769-b2c5dfa96eef admin admin] PATH_INFO: `/v3/projects/90e272e295fb4aa7af0d4d3e76ccad79` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:52.793708 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbda0823-9e9c-4423-8769-b2c5dfa96eef admin admin] RBAC: Authorizing `identity:delete_project(project_id=90e272e295fb4aa7af0d4d3e76ccad79)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:52.795677 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbda0823-9e9c-4423-8769-b2c5dfa96eef admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:52.845180 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-fbda0823-9e9c-4423-8769-b2c5dfa96eef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90e272e295fb4aa7af0d4d3e76ccad79', 'request_id': 'req-fbda0823-9e9c-4423-8769-b2c5dfa96eef'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:52.855026 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-fbda0823-9e9c-4423-8769-b2c5dfa96eef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90e272e295fb4aa7af0d4d3e76ccad79', 'request_id': 'req-fbda0823-9e9c-4423-8769-b2c5dfa96eef'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:52.860706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-fbda0823-9e9c-4423-8769-b2c5dfa96eef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90e272e295fb4aa7af0d4d3e76ccad79', 'request_id': 'req-fbda0823-9e9c-4423-8769-b2c5dfa96eef'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:52.872729 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-fbda0823-9e9c-4423-8769-b2c5dfa96eef admin admin] The token cache is being invalidate because project 90e272e295fb4aa7af0d4d3e76ccad79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:52.873013 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-fbda0823-9e9c-4423-8769-b2c5dfa96eef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:52.873697 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-fbda0823-9e9c-4423-8769-b2c5dfa96eef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:52.875333 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 867/1731] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:52 2022] DELETE /identity/v3/projects/90e272e295fb4aa7af0d4d3e76ccad79 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:52.881018 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c57fc-784d-4b00-92ec-d1f44c2589d2 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:52.881292 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c57fc-784d-4b00-92ec-d1f44c2589d2 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:52.881554 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c57fc-784d-4b00-92ec-d1f44c2589d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:52.903246 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-9c8c57fc-784d-4b00-92ec-d1f44c2589d2 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:52.904440 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-9c8c57fc-784d-4b00-92ec-d1f44c2589d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:52.967784 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 865/1732] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:52.975667 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eba742-35b0-4f29-acb9-308ae4939f20 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:52.976002 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eba742-35b0-4f29-acb9-308ae4939f20 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:52.976247 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eba742-35b0-4f29-acb9-308ae4939f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:52.997255 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-30eba742-35b0-4f29-acb9-308ae4939f20 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:52.998478 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-30eba742-35b0-4f29-acb9-308ae4939f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:53.007787 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 868/1733] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:53.012748 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c57fc-784d-4b00-92ec-d1f44c2589d2 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:53.024742 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c57fc-784d-4b00-92ec-d1f44c2589d2 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:53.029875 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac569ae1-02a8-492f-af38-0190ca5fd18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:53.030480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac569ae1-02a8-492f-af38-0190ca5fd18f admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:53.030791 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac569ae1-02a8-492f-af38-0190ca5fd18f admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:53.031060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac569ae1-02a8-492f-af38-0190ca5fd18f admin admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:53.031697 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac569ae1-02a8-492f-af38-0190ca5fd18f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:53.031998 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac569ae1-02a8-492f-af38-0190ca5fd18f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:53.033590 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac569ae1-02a8-492f-af38-0190ca5fd18f admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:53.046897 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 866/1734] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:28:53.051570 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eba742-35b0-4f29-acb9-308ae4939f20 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:53.061744 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eba742-35b0-4f29-acb9-308ae4939f20 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:53.067689 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af651c-79e9-4df6-afff-b5b516399cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:53.068308 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af651c-79e9-4df6-afff-b5b516399cf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:53.068603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af651c-79e9-4df6-afff-b5b516399cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:53.068876 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af651c-79e9-4df6-afff-b5b516399cf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:53.069545 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af651c-79e9-4df6-afff-b5b516399cf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:53.071504 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af651c-79e9-4df6-afff-b5b516399cf0 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:53.104852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 869/1735] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:53 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:53.109262 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac569ae1-02a8-492f-af38-0190ca5fd18f admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:53.121029 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac569ae1-02a8-492f-af38-0190ca5fd18f admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:53.180054 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8086d6c-85e3-4ede-8404-ee2af23609a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:53.180652 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8086d6c-85e3-4ede-8404-ee2af23609a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:53.180920 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8086d6c-85e3-4ede-8404-ee2af23609a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:53.181246 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8086d6c-85e3-4ede-8404-ee2af23609a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:53.181950 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8086d6c-85e3-4ede-8404-ee2af23609a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:53.183909 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8086d6c-85e3-4ede-8404-ee2af23609a1 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:53.206568 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 867/1736] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:53 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:53.210714 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af651c-79e9-4df6-afff-b5b516399cf0 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:53.221285 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af651c-79e9-4df6-afff-b5b516399cf0 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:53.226147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6242b3-7a25-4d90-848c-23b1cd7fb986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:53.226796 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6242b3-7a25-4d90-848c-23b1cd7fb986 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:53.227077 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6242b3-7a25-4d90-848c-23b1cd7fb986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:53.227350 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6242b3-7a25-4d90-848c-23b1cd7fb986 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:53.228064 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6242b3-7a25-4d90-848c-23b1cd7fb986 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:53.229863 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6242b3-7a25-4d90-848c-23b1cd7fb986 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:53.240547 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 870/1737] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:53.245188 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8086d6c-85e3-4ede-8404-ee2af23609a1 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:53.255756 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8086d6c-85e3-4ede-8404-ee2af23609a1 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:53.260963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733fb5fb-3c40-447c-b005-07ec2d07b1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:53.261574 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fb5fb-3c40-447c-b005-07ec2d07b1e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:53.261873 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fb5fb-3c40-447c-b005-07ec2d07b1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:53.262109 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fb5fb-3c40-447c-b005-07ec2d07b1e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:53.262778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733fb5fb-3c40-447c-b005-07ec2d07b1e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:53.264358 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733fb5fb-3c40-447c-b005-07ec2d07b1e7 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:53.277249 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 868/1738] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:53.281556 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6242b3-7a25-4d90-848c-23b1cd7fb986 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:53.292592 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6242b3-7a25-4d90-848c-23b1cd7fb986 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:53.298069 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff9ba6c-0a32-44d1-aaf1-1b90ed53da5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:53.299110 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9ba6c-0a32-44d1-aaf1-1b90ed53da5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:53.299391 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9ba6c-0a32-44d1-aaf1-1b90ed53da5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:53.299664 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9ba6c-0a32-44d1-aaf1-1b90ed53da5b admin admin] PATH_INFO: `/v3/projects/f619d4a5d383409db5b71c59d7c0d274/users/05922a9f879c4c1c8c93ea12825306b3/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:53.315771 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff9ba6c-0a32-44d1-aaf1-1b90ed53da5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f619d4a5d383409db5b71c59d7c0d274, user_id=05922a9f879c4c1c8c93ea12825306b3, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:53.318496 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff9ba6c-0a32-44d1-aaf1-1b90ed53da5b admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:53.335664 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 871/1739] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:53 2022] PUT /identity/v3/projects/f619d4a5d383409db5b71c59d7c0d274/users/05922a9f879c4c1c8c93ea12825306b3/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:53.346444 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d2b68-310f-48df-be05-b76f22aebcf6 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:53.346759 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d2b68-310f-48df-be05-b76f22aebcf6 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:53.347024 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d2b68-310f-48df-be05-b76f22aebcf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:53.382778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-c19d2b68-310f-48df-be05-b76f22aebcf6 None None] MFA Rules not processed for user `05922a9f879c4c1c8c93ea12825306b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:53.384132 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-c19d2b68-310f-48df-be05-b76f22aebcf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:53.408185 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:53.451487 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 869/1740] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:53.456334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff9ba6c-0a32-44d1-aaf1-1b90ed53da5b admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:53.531667 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff9ba6c-0a32-44d1-aaf1-1b90ed53da5b admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:53.536454 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd486d2e-acd6-492e-b7ae-8bffff2f4834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:53.537054 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd486d2e-acd6-492e-b7ae-8bffff2f4834 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:53.537335 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd486d2e-acd6-492e-b7ae-8bffff2f4834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:53.537596 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd486d2e-acd6-492e-b7ae-8bffff2f4834 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:53.538224 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd486d2e-acd6-492e-b7ae-8bffff2f4834 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:53.540148 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd486d2e-acd6-492e-b7ae-8bffff2f4834 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:53.550269 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:53.571633 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 872/1741] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:53 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:28:53.576158 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d2b68-310f-48df-be05-b76f22aebcf6 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:53.587024 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d2b68-310f-48df-be05-b76f22aebcf6 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:53.648749 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbd9a6-e04a-4fac-b6fe-f23b478b3eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:53.649361 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbd9a6-e04a-4fac-b6fe-f23b478b3eb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:53.649630 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbd9a6-e04a-4fac-b6fe-f23b478b3eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:53.649934 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbd9a6-e04a-4fac-b6fe-f23b478b3eb8 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:53.650916 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bbd9a6-e04a-4fac-b6fe-f23b478b3eb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:53.652831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bbd9a6-e04a-4fac-b6fe-f23b478b3eb8 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:53.677843 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 870/1742] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:53 2022] POST /identity/v3/users => generated 298 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:28:53.680935 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd486d2e-acd6-492e-b7ae-8bffff2f4834 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:53.691701 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd486d2e-acd6-492e-b7ae-8bffff2f4834 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:53.696557 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69a67e-c109-4bc7-b576-b96d5f2be86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:53.697468 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69a67e-c109-4bc7-b576-b96d5f2be86e admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:53.697735 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69a67e-c109-4bc7-b576-b96d5f2be86e admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:53.698032 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69a67e-c109-4bc7-b576-b96d5f2be86e admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:53.698682 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c69a67e-c109-4bc7-b576-b96d5f2be86e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:53.700296 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c69a67e-c109-4bc7-b576-b96d5f2be86e admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:53.713868 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 873/1743] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:53.715321 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbd9a6-e04a-4fac-b6fe-f23b478b3eb8 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:53.725298 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbd9a6-e04a-4fac-b6fe-f23b478b3eb8 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:53.730164 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc738b4d-e371-4ce9-a822-a34be1fccea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:53.730851 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc738b4d-e371-4ce9-a822-a34be1fccea6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:53.731111 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc738b4d-e371-4ce9-a822-a34be1fccea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:53.731379 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc738b4d-e371-4ce9-a822-a34be1fccea6 admin admin] PATH_INFO: `/v3/projects/5362636c4b334c2c83210a8db41010ac/users/9882c0a7ec5047f0a1253f2ef1a4e7c4/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:53.746586 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc738b4d-e371-4ce9-a822-a34be1fccea6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5362636c4b334c2c83210a8db41010ac, user_id=9882c0a7ec5047f0a1253f2ef1a4e7c4, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:53.749333 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc738b4d-e371-4ce9-a822-a34be1fccea6 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:53.765243 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 871/1744] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:53 2022] PUT /identity/v3/projects/5362636c4b334c2c83210a8db41010ac/users/9882c0a7ec5047f0a1253f2ef1a4e7c4/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:53.769250 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69a67e-c109-4bc7-b576-b96d5f2be86e admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:53.779735 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69a67e-c109-4bc7-b576-b96d5f2be86e admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:53.851094 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce0c05-3dcf-4498-99b8-6e16ed8dc7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:53.851765 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce0c05-3dcf-4498-99b8-6e16ed8dc7dc admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:53.852086 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce0c05-3dcf-4498-99b8-6e16ed8dc7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:53.852360 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce0c05-3dcf-4498-99b8-6e16ed8dc7dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:53.853129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ce0c05-3dcf-4498-99b8-6e16ed8dc7dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:53.854982 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ce0c05-3dcf-4498-99b8-6e16ed8dc7dc admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:53.865228 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 874/1745] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:53.869961 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc738b4d-e371-4ce9-a822-a34be1fccea6 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:53.881247 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc738b4d-e371-4ce9-a822-a34be1fccea6 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:53.887230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f6cae-b2b2-4241-8121-170329c46607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:53.887886 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f6cae-b2b2-4241-8121-170329c46607 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:53.888167 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f6cae-b2b2-4241-8121-170329c46607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:53.888427 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f6cae-b2b2-4241-8121-170329c46607 admin admin] PATH_INFO: `/v3/projects/5362636c4b334c2c83210a8db41010ac/users/9882c0a7ec5047f0a1253f2ef1a4e7c4/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:53.890623 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99f6cae-b2b2-4241-8121-170329c46607 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5362636c4b334c2c83210a8db41010ac, user_id=9882c0a7ec5047f0a1253f2ef1a4e7c4, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:53.893290 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99f6cae-b2b2-4241-8121-170329c46607 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:53.905209 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 872/1746] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:53 2022] PUT /identity/v3/projects/5362636c4b334c2c83210a8db41010ac/users/9882c0a7ec5047f0a1253f2ef1a4e7c4/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:53.922318 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a147f-a618-4856-9d1b-d05108b794ed None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:53.922700 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a147f-a618-4856-9d1b-d05108b794ed None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:53.923066 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a147f-a618-4856-9d1b-d05108b794ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:53.955290 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-e95a147f-a618-4856-9d1b-d05108b794ed None None] MFA Rules not processed for user `9882c0a7ec5047f0a1253f2ef1a4e7c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:53.956495 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-e95a147f-a618-4856-9d1b-d05108b794ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:54.026375 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 875/1747] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:54.037899 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f6cae-b2b2-4241-8121-170329c46607 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:54.040115 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-f99f6cae-b2b2-4241-8121-170329c46607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:54.101465 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f6cae-b2b2-4241-8121-170329c46607 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:54.105623 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b64c58-7905-4e40-bb9d-bd2716e7054e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:54.106232 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b64c58-7905-4e40-bb9d-bd2716e7054e service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:54.106501 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b64c58-7905-4e40-bb9d-bd2716e7054e service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:54.106798 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b64c58-7905-4e40-bb9d-bd2716e7054e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:54.117095 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b64c58-7905-4e40-bb9d-bd2716e7054e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:54.118596 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b64c58-7905-4e40-bb9d-bd2716e7054e service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:54.129307 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 873/1748] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:54 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:54.229523 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a147f-a618-4856-9d1b-d05108b794ed None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:54.240001 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a147f-a618-4856-9d1b-d05108b794ed None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:54.244343 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d09bef-9c21-42cb-964c-1654d92fd8ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:54.245079 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d09bef-9c21-42cb-964c-1654d92fd8ea service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:54.245357 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d09bef-9c21-42cb-964c-1654d92fd8ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:54.245628 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d09bef-9c21-42cb-964c-1654d92fd8ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:54.308879 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d09bef-9c21-42cb-964c-1654d92fd8ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:54.310425 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d09bef-9c21-42cb-964c-1654d92fd8ea service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:54.314221 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:54.324794 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 876/1749] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:54.637781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b64c58-7905-4e40-bb9d-bd2716e7054e service cinder] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:54.665487 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:54.717434 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b64c58-7905-4e40-bb9d-bd2716e7054e service cinder] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:54.721992 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d422b-acb0-454b-bc40-31b23fbd23eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:54.722632 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d422b-acb0-454b-bc40-31b23fbd23eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:54.722939 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d422b-acb0-454b-bc40-31b23fbd23eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:54.723205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d422b-acb0-454b-bc40-31b23fbd23eb admin admin] PATH_INFO: `/v3/users/05922a9f879c4c1c8c93ea12825306b3` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:54.724304 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483d422b-acb0-454b-bc40-31b23fbd23eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=05922a9f879c4c1c8c93ea12825306b3)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:54.726235 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483d422b-acb0-454b-bc40-31b23fbd23eb admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:54.780129 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-483d422b-acb0-454b-bc40-31b23fbd23eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05922a9f879c4c1c8c93ea12825306b3'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:54.780799 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-483d422b-acb0-454b-bc40-31b23fbd23eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05922a9f879c4c1c8c93ea12825306b3'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:54.787951 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-483d422b-acb0-454b-bc40-31b23fbd23eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05922a9f879c4c1c8c93ea12825306b3'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:54.801474 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-483d422b-acb0-454b-bc40-31b23fbd23eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05922a9f879c4c1c8c93ea12825306b3'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:54.812187 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-483d422b-acb0-454b-bc40-31b23fbd23eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05922a9f879c4c1c8c93ea12825306b3'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:54.819047 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 874/1750] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:54 2022] DELETE /identity/v3/users/05922a9f879c4c1c8c93ea12825306b3 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:54.823752 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473b38c-8d03-4580-a851-53288f512e34 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:54.824471 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473b38c-8d03-4580-a851-53288f512e34 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:54.824879 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473b38c-8d03-4580-a851-53288f512e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:54.862223 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-4473b38c-8d03-4580-a851-53288f512e34 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:54.863510 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-4473b38c-8d03-4580-a851-53288f512e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:54.923359 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 877/1751] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:54.933522 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d422b-acb0-454b-bc40-31b23fbd23eb admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:54.935062 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-483d422b-acb0-454b-bc40-31b23fbd23eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:54.993608 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d422b-acb0-454b-bc40-31b23fbd23eb admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:54.997843 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15bdd41-0f55-44df-b1e4-f81e23fdffe9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:54.998458 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15bdd41-0f55-44df-b1e4-f81e23fdffe9 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:54.998757 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15bdd41-0f55-44df-b1e4-f81e23fdffe9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:54.999027 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15bdd41-0f55-44df-b1e4-f81e23fdffe9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:55.009285 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15bdd41-0f55-44df-b1e4-f81e23fdffe9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:55.010972 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15bdd41-0f55-44df-b1e4-f81e23fdffe9 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:55.020591 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 875/1752] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:28:55.343547 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473b38c-8d03-4580-a851-53288f512e34 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:55.345130 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-4473b38c-8d03-4580-a851-53288f512e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:55.358440 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473b38c-8d03-4580-a851-53288f512e34 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:55.364220 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1b867-b3f8-4543-a25e-b8803b03d843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:55.364863 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1b867-b3f8-4543-a25e-b8803b03d843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:55.365136 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1b867-b3f8-4543-a25e-b8803b03d843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:55.365405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1b867-b3f8-4543-a25e-b8803b03d843 admin admin] PATH_INFO: `/v3/projects/f619d4a5d383409db5b71c59d7c0d274` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:55.366647 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1b867-b3f8-4543-a25e-b8803b03d843 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f619d4a5d383409db5b71c59d7c0d274)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:55.368791 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1b867-b3f8-4543-a25e-b8803b03d843 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:55.424404 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-23f1b867-b3f8-4543-a25e-b8803b03d843 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f619d4a5d383409db5b71c59d7c0d274', 'request_id': 'req-23f1b867-b3f8-4543-a25e-b8803b03d843'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:55.434351 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-23f1b867-b3f8-4543-a25e-b8803b03d843 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f619d4a5d383409db5b71c59d7c0d274', 'request_id': 'req-23f1b867-b3f8-4543-a25e-b8803b03d843'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:55.440383 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-23f1b867-b3f8-4543-a25e-b8803b03d843 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f619d4a5d383409db5b71c59d7c0d274', 'request_id': 'req-23f1b867-b3f8-4543-a25e-b8803b03d843'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:55.452165 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-23f1b867-b3f8-4543-a25e-b8803b03d843 admin admin] The token cache is being invalidate because project f619d4a5d383409db5b71c59d7c0d274 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:55.452452 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-23f1b867-b3f8-4543-a25e-b8803b03d843 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:55.453093 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-23f1b867-b3f8-4543-a25e-b8803b03d843 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:55.454598 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 878/1753] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:55 2022] DELETE /identity/v3/projects/f619d4a5d383409db5b71c59d7c0d274 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:55.458794 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15bdd41-0f55-44df-b1e4-f81e23fdffe9 service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:55.468624 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15bdd41-0f55-44df-b1e4-f81e23fdffe9 service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:55.514743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:55.535655 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550988d-a1b0-4ba4-a22d-10d9ea5892fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:55.536298 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550988d-a1b0-4ba4-a22d-10d9ea5892fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:55.536572 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550988d-a1b0-4ba4-a22d-10d9ea5892fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:55.536839 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550988d-a1b0-4ba4-a22d-10d9ea5892fa admin admin] PATH_INFO: `/v3/users/9882c0a7ec5047f0a1253f2ef1a4e7c4` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:55.537967 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550988d-a1b0-4ba4-a22d-10d9ea5892fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=9882c0a7ec5047f0a1253f2ef1a4e7c4)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:55.539947 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550988d-a1b0-4ba4-a22d-10d9ea5892fa admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:55.598301 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-0550988d-a1b0-4ba4-a22d-10d9ea5892fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9882c0a7ec5047f0a1253f2ef1a4e7c4'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:55.598896 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-0550988d-a1b0-4ba4-a22d-10d9ea5892fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9882c0a7ec5047f0a1253f2ef1a4e7c4'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:55.605993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-0550988d-a1b0-4ba4-a22d-10d9ea5892fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9882c0a7ec5047f0a1253f2ef1a4e7c4'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:55.619141 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-0550988d-a1b0-4ba4-a22d-10d9ea5892fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9882c0a7ec5047f0a1253f2ef1a4e7c4'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:55.629112 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-0550988d-a1b0-4ba4-a22d-10d9ea5892fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9882c0a7ec5047f0a1253f2ef1a4e7c4'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:55.635441 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 876/1754] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:28:55 2022] DELETE /identity/v3/users/9882c0a7ec5047f0a1253f2ef1a4e7c4 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:28:56.051916 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1b867-b3f8-4543-a25e-b8803b03d843 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:56.053762 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-23f1b867-b3f8-4543-a25e-b8803b03d843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:56.069155 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1b867-b3f8-4543-a25e-b8803b03d843 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:56.127709 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7590b-6116-46bd-b44c-f49e3006c8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:56.128350 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7590b-6116-46bd-b44c-f49e3006c8c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:56.128626 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7590b-6116-46bd-b44c-f49e3006c8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:56.128894 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7590b-6116-46bd-b44c-f49e3006c8c5 admin admin] PATH_INFO: `/v3/projects/5362636c4b334c2c83210a8db41010ac` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:56.130316 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac7590b-6116-46bd-b44c-f49e3006c8c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5362636c4b334c2c83210a8db41010ac)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:56.132265 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac7590b-6116-46bd-b44c-f49e3006c8c5 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:56.157551 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:56.187644 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-bac7590b-6116-46bd-b44c-f49e3006c8c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5362636c4b334c2c83210a8db41010ac', 'request_id': 'req-bac7590b-6116-46bd-b44c-f49e3006c8c5'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:56.198322 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-bac7590b-6116-46bd-b44c-f49e3006c8c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5362636c4b334c2c83210a8db41010ac', 'request_id': 'req-bac7590b-6116-46bd-b44c-f49e3006c8c5'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:56.205015 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-bac7590b-6116-46bd-b44c-f49e3006c8c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5362636c4b334c2c83210a8db41010ac', 'request_id': 'req-bac7590b-6116-46bd-b44c-f49e3006c8c5'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:56.217574 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-bac7590b-6116-46bd-b44c-f49e3006c8c5 admin admin] The token cache is being invalidate because project 5362636c4b334c2c83210a8db41010ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:28:56.217868 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-bac7590b-6116-46bd-b44c-f49e3006c8c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:56.218509 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-bac7590b-6116-46bd-b44c-f49e3006c8c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:28:56.220546 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 879/1755] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:28:56 2022] DELETE /identity/v3/projects/5362636c4b334c2c83210a8db41010ac => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:56.226819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e137f16-43a4-4984-ab72-d97ed04045fa None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:56.227281 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e137f16-43a4-4984-ab72-d97ed04045fa None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:56.227500 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e137f16-43a4-4984-ab72-d97ed04045fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:56.247414 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-1e137f16-43a4-4984-ab72-d97ed04045fa None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:56.248561 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-1e137f16-43a4-4984-ab72-d97ed04045fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:56.320178 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 877/1756] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:56.329133 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec12006-abb9-46ff-be81-732b37b554d1 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:56.329486 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec12006-abb9-46ff-be81-732b37b554d1 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:56.329882 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec12006-abb9-46ff-be81-732b37b554d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:56.353389 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-7ec12006-abb9-46ff-be81-732b37b554d1 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:56.354551 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-7ec12006-abb9-46ff-be81-732b37b554d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:56.362884 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 880/1757] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:28:56.367465 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e137f16-43a4-4984-ab72-d97ed04045fa None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:56.378582 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e137f16-43a4-4984-ab72-d97ed04045fa None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:56.383511 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c8e96-b23b-48f0-9ddc-a9a07dcbc538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:56.384096 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c8e96-b23b-48f0-9ddc-a9a07dcbc538 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:56.384392 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c8e96-b23b-48f0-9ddc-a9a07dcbc538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:56.384683 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c8e96-b23b-48f0-9ddc-a9a07dcbc538 admin admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:56.385325 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c8e96-b23b-48f0-9ddc-a9a07dcbc538 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:28:56.385626 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c8e96-b23b-48f0-9ddc-a9a07dcbc538 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:56.387267 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c8e96-b23b-48f0-9ddc-a9a07dcbc538 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:56.400181 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 878/1758] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:28:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:28:56.406628 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec12006-abb9-46ff-be81-732b37b554d1 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:56.419230 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec12006-abb9-46ff-be81-732b37b554d1 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:56.424132 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb90d5-0181-4d26-a182-dd6ed3814044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:56.424728 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb90d5-0181-4d26-a182-dd6ed3814044 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:56.425004 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb90d5-0181-4d26-a182-dd6ed3814044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:56.425272 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb90d5-0181-4d26-a182-dd6ed3814044 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:56.425893 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb90d5-0181-4d26-a182-dd6ed3814044 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:56.427823 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb90d5-0181-4d26-a182-dd6ed3814044 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:56.458381 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 881/1759] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:28:56 2022] POST /identity/v3/projects => generated 389 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:56.462725 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c8e96-b23b-48f0-9ddc-a9a07dcbc538 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:56.472718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c8e96-b23b-48f0-9ddc-a9a07dcbc538 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:56.518426 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:28:56.533376 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406011d7-b48f-41c0-b6d9-eda713565120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:56.534280 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406011d7-b48f-41c0-b6d9-eda713565120 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:56.534830 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406011d7-b48f-41c0-b6d9-eda713565120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:56.535278 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406011d7-b48f-41c0-b6d9-eda713565120 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:56.536187 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406011d7-b48f-41c0-b6d9-eda713565120 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:56.539373 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406011d7-b48f-41c0-b6d9-eda713565120 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:56.563437 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 879/1760] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:28:56 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:28:56.568806 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb90d5-0181-4d26-a182-dd6ed3814044 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:56.582184 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb90d5-0181-4d26-a182-dd6ed3814044 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:56.588684 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a5e3f8-f0b5-49b4-93a3-730fd9a4a1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:56.589415 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5e3f8-f0b5-49b4-93a3-730fd9a4a1e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:56.591280 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5e3f8-f0b5-49b4-93a3-730fd9a4a1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:56.591613 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5e3f8-f0b5-49b4-93a3-730fd9a4a1e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:56.592354 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a5e3f8-f0b5-49b4-93a3-730fd9a4a1e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:56.594051 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a5e3f8-f0b5-49b4-93a3-730fd9a4a1e8 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:56.608881 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 882/1761] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:56.610186 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406011d7-b48f-41c0-b6d9-eda713565120 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:56.623738 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406011d7-b48f-41c0-b6d9-eda713565120 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:56.629209 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c975e-6430-4b6b-8bbe-746775defe6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:56.629775 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c975e-6430-4b6b-8bbe-746775defe6b admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:56.630054 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c975e-6430-4b6b-8bbe-746775defe6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:56.630328 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c975e-6430-4b6b-8bbe-746775defe6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:56.631016 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c975e-6430-4b6b-8bbe-746775defe6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:56.632937 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c975e-6430-4b6b-8bbe-746775defe6b admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:56.650701 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 880/1762] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:28:56.652523 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a5e3f8-f0b5-49b4-93a3-730fd9a4a1e8 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:56.667428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a5e3f8-f0b5-49b4-93a3-730fd9a4a1e8 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:56.674026 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469671a5-ed54-4cdb-a2dc-c93a0902fca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:56.674221 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469671a5-ed54-4cdb-a2dc-c93a0902fca0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:56.674467 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469671a5-ed54-4cdb-a2dc-c93a0902fca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:56.674812 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469671a5-ed54-4cdb-a2dc-c93a0902fca0 admin admin] PATH_INFO: `/v3/projects/d2b4884c02874086acc5bf75e1da7ac4/users/04940315f596486ebbf4bc1d824a5c42/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:56.692464 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469671a5-ed54-4cdb-a2dc-c93a0902fca0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2b4884c02874086acc5bf75e1da7ac4, user_id=04940315f596486ebbf4bc1d824a5c42, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:56.695255 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469671a5-ed54-4cdb-a2dc-c93a0902fca0 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:56.720830 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 883/1763] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:28:56 2022] PUT /identity/v3/projects/d2b4884c02874086acc5bf75e1da7ac4/users/04940315f596486ebbf4bc1d824a5c42/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:28:56.722961 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62728e-4a97-486a-afdc-c8e0914b6847 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:56.723081 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62728e-4a97-486a-afdc-c8e0914b6847 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:56.723318 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62728e-4a97-486a-afdc-c8e0914b6847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:56.757477 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-0c62728e-4a97-486a-afdc-c8e0914b6847 None None] MFA Rules not processed for user `04940315f596486ebbf4bc1d824a5c42`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:28:56.758897 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-0c62728e-4a97-486a-afdc-c8e0914b6847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:56.834205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 881/1764] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:28:56 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:28:56.846067 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469671a5-ed54-4cdb-a2dc-c93a0902fca0 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:28:56.849184 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-469671a5-ed54-4cdb-a2dc-c93a0902fca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:28:56.915686 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469671a5-ed54-4cdb-a2dc-c93a0902fca0 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:28:56.919827 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ad386-8a52-4bac-b6f9-f856ae625565 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:28:56.920433 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ad386-8a52-4bac-b6f9-f856ae625565 service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:28:56.920703 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ad386-8a52-4bac-b6f9-f856ae625565 service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:28:56.920966 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ad386-8a52-4bac-b6f9-f856ae625565 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:28:56.929983 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512ad386-8a52-4bac-b6f9-f856ae625565 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:28:56.931498 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512ad386-8a52-4bac-b6f9-f856ae625565 service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:28:56.940902 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 884/1765] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:28:56 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:03.617688 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62728e-4a97-486a-afdc-c8e0914b6847 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:03.619660 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-0c62728e-4a97-486a-afdc-c8e0914b6847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:03.636576 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62728e-4a97-486a-afdc-c8e0914b6847 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:03.641118 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f3afa7-1669-43b5-bcf9-72736f5c8c41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:03.641904 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3afa7-1669-43b5-bcf9-72736f5c8c41 service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:03.642535 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3afa7-1669-43b5-bcf9-72736f5c8c41 service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:03.642535 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3afa7-1669-43b5-bcf9-72736f5c8c41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:03.646865 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-c5f3afa7-1669-43b5-bcf9-72736f5c8c41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:03.731450 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f3afa7-1669-43b5-bcf9-72736f5c8c41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:03.733000 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f3afa7-1669-43b5-bcf9-72736f5c8c41 service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:03.743241 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 882/1766] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:03 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:06.936673 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=68,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:13.738372 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=74,keystone:INSERT=3 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:14.283542 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ad386-8a52-4bac-b6f9-f856ae625565 service cinder] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:14.285022 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-512ad386-8a52-4bac-b6f9-f856ae625565 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:14.357109 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ad386-8a52-4bac-b6f9-f856ae625565 service cinder] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:14.361730 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8fecfb-300f-4a88-95b4-b726eb5dc8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:14.362370 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fecfb-300f-4a88-95b4-b726eb5dc8b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:14.362643 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fecfb-300f-4a88-95b4-b726eb5dc8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:14.362944 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fecfb-300f-4a88-95b4-b726eb5dc8b2 admin admin] PATH_INFO: `/v3/users/d1ab3a393c3e4e1eb0bd6865f7080548` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:14.364064 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8fecfb-300f-4a88-95b4-b726eb5dc8b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1ab3a393c3e4e1eb0bd6865f7080548)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:14.365998 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8fecfb-300f-4a88-95b4-b726eb5dc8b2 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:14.419016 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-3b8fecfb-300f-4a88-95b4-b726eb5dc8b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1ab3a393c3e4e1eb0bd6865f7080548'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:14.429451 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-3b8fecfb-300f-4a88-95b4-b726eb5dc8b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1ab3a393c3e4e1eb0bd6865f7080548'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:14.430277 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-3b8fecfb-300f-4a88-95b4-b726eb5dc8b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1ab3a393c3e4e1eb0bd6865f7080548'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:14.462477 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-3b8fecfb-300f-4a88-95b4-b726eb5dc8b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1ab3a393c3e4e1eb0bd6865f7080548'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:14.485174 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-3b8fecfb-300f-4a88-95b4-b726eb5dc8b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1ab3a393c3e4e1eb0bd6865f7080548'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:14.492298 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 885/1767] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:14 2022] DELETE /identity/v3/users/d1ab3a393c3e4e1eb0bd6865f7080548 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:14.688109 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f3afa7-1669-43b5-bcf9-72736f5c8c41 service cinder] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:14.699421 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f3afa7-1669-43b5-bcf9-72736f5c8c41 service cinder] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:14.772601 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff308c-4f42-49da-a1bf-6f73d329947e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:14.773284 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff308c-4f42-49da-a1bf-6f73d329947e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:14.773528 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff308c-4f42-49da-a1bf-6f73d329947e admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:14.773795 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff308c-4f42-49da-a1bf-6f73d329947e admin admin] PATH_INFO: `/v3/projects/20c55579c8b44160ba38f1d5337c64c2` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:14.774951 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ff308c-4f42-49da-a1bf-6f73d329947e admin admin] RBAC: Authorizing `identity:delete_project(project_id=20c55579c8b44160ba38f1d5337c64c2)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:14.776857 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ff308c-4f42-49da-a1bf-6f73d329947e admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:14.825790 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-68ff308c-4f42-49da-a1bf-6f73d329947e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20c55579c8b44160ba38f1d5337c64c2', 'request_id': 'req-68ff308c-4f42-49da-a1bf-6f73d329947e'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:14.832383 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-68ff308c-4f42-49da-a1bf-6f73d329947e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20c55579c8b44160ba38f1d5337c64c2', 'request_id': 'req-68ff308c-4f42-49da-a1bf-6f73d329947e'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:14.838046 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-68ff308c-4f42-49da-a1bf-6f73d329947e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20c55579c8b44160ba38f1d5337c64c2', 'request_id': 'req-68ff308c-4f42-49da-a1bf-6f73d329947e'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:14.851378 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-68ff308c-4f42-49da-a1bf-6f73d329947e admin admin] The token cache is being invalidate because project 20c55579c8b44160ba38f1d5337c64c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:14.851658 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-68ff308c-4f42-49da-a1bf-6f73d329947e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:14.852257 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-68ff308c-4f42-49da-a1bf-6f73d329947e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:14.853744 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 883/1768] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:14 2022] DELETE /identity/v3/projects/20c55579c8b44160ba38f1d5337c64c2 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:14.857824 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8fecfb-300f-4a88-95b4-b726eb5dc8b2 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:14.859395 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-3b8fecfb-300f-4a88-95b4-b726eb5dc8b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:14.872377 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8fecfb-300f-4a88-95b4-b726eb5dc8b2 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:14.934263 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d491437-69d6-4c2e-9c36-b0473ca65ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:14.935086 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d491437-69d6-4c2e-9c36-b0473ca65ac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:14.935369 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d491437-69d6-4c2e-9c36-b0473ca65ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:14.935776 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d491437-69d6-4c2e-9c36-b0473ca65ac1 admin admin] PATH_INFO: `/v3/users/12e2a78ac61d4749ab8e20647e009c85` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:14.937061 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d491437-69d6-4c2e-9c36-b0473ca65ac1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12e2a78ac61d4749ab8e20647e009c85)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:14.939162 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d491437-69d6-4c2e-9c36-b0473ca65ac1 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:14.965031 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:15.007524 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-2d491437-69d6-4c2e-9c36-b0473ca65ac1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12e2a78ac61d4749ab8e20647e009c85'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:15.015170 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-2d491437-69d6-4c2e-9c36-b0473ca65ac1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12e2a78ac61d4749ab8e20647e009c85'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:15.015916 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-2d491437-69d6-4c2e-9c36-b0473ca65ac1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12e2a78ac61d4749ab8e20647e009c85'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:15.028206 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-2d491437-69d6-4c2e-9c36-b0473ca65ac1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12e2a78ac61d4749ab8e20647e009c85'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:15.037600 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-2d491437-69d6-4c2e-9c36-b0473ca65ac1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12e2a78ac61d4749ab8e20647e009c85'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:15.043614 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 886/1769] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:14 2022] DELETE /identity/v3/users/12e2a78ac61d4749ab8e20647e009c85 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:15.229393 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff308c-4f42-49da-a1bf-6f73d329947e admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:15.231354 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-68ff308c-4f42-49da-a1bf-6f73d329947e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:15.245767 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff308c-4f42-49da-a1bf-6f73d329947e admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:15.302112 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d491437-69d6-4c2e-9c36-b0473ca65ac1 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:15.304471 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-2d491437-69d6-4c2e-9c36-b0473ca65ac1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:15.319102 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc311850-7eb9-4229-b926-3e645fe654f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:15.319755 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc311850-7eb9-4229-b926-3e645fe654f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:15.320249 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc311850-7eb9-4229-b926-3e645fe654f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:15.320528 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc311850-7eb9-4229-b926-3e645fe654f5 admin admin] PATH_INFO: `/v3/projects/86f2720035c54fefb0c1d4bb47661361` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:15.321701 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc311850-7eb9-4229-b926-3e645fe654f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86f2720035c54fefb0c1d4bb47661361)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:15.324728 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc311850-7eb9-4229-b926-3e645fe654f5 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:15.367669 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:15.387969 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-cc311850-7eb9-4229-b926-3e645fe654f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86f2720035c54fefb0c1d4bb47661361', 'request_id': 'req-cc311850-7eb9-4229-b926-3e645fe654f5'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:15.395234 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-cc311850-7eb9-4229-b926-3e645fe654f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86f2720035c54fefb0c1d4bb47661361', 'request_id': 'req-cc311850-7eb9-4229-b926-3e645fe654f5'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:15.397930 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d491437-69d6-4c2e-9c36-b0473ca65ac1 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:15.401461 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-cc311850-7eb9-4229-b926-3e645fe654f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86f2720035c54fefb0c1d4bb47661361', 'request_id': 'req-cc311850-7eb9-4229-b926-3e645fe654f5'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:15.416481 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-cc311850-7eb9-4229-b926-3e645fe654f5 admin admin] The token cache is being invalidate because project 86f2720035c54fefb0c1d4bb47661361 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:15.417029 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-cc311850-7eb9-4229-b926-3e645fe654f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:15.419101 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-cc311850-7eb9-4229-b926-3e645fe654f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:15.423312 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 884/1770] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:15 2022] DELETE /identity/v3/projects/86f2720035c54fefb0c1d4bb47661361 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:15.429265 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a88141-c75c-4b0d-ba80-b8c9471f3dfb None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:15.429733 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a88141-c75c-4b0d-ba80-b8c9471f3dfb None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:15.430700 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a88141-c75c-4b0d-ba80-b8c9471f3dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:15.453266 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-f6a88141-c75c-4b0d-ba80-b8c9471f3dfb None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:15.454620 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-f6a88141-c75c-4b0d-ba80-b8c9471f3dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:15.486742 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f9caa7-b57a-4813-9f5a-eff8a1dfce80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:15.487522 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9caa7-b57a-4813-9f5a-eff8a1dfce80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:15.487862 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9caa7-b57a-4813-9f5a-eff8a1dfce80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:15.488329 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9caa7-b57a-4813-9f5a-eff8a1dfce80 admin admin] PATH_INFO: `/v3/users/1bf9cc586c5b4c9cb501939c587d6a6e` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:15.489754 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f9caa7-b57a-4813-9f5a-eff8a1dfce80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bf9cc586c5b4c9cb501939c587d6a6e)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:15.493197 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f9caa7-b57a-4813-9f5a-eff8a1dfce80 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:15.573139 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-b8f9caa7-b57a-4813-9f5a-eff8a1dfce80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bf9cc586c5b4c9cb501939c587d6a6e'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:15.574345 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:15.584462 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 885/1771] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:15.585123 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-b8f9caa7-b57a-4813-9f5a-eff8a1dfce80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bf9cc586c5b4c9cb501939c587d6a6e'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:15.585896 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-b8f9caa7-b57a-4813-9f5a-eff8a1dfce80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bf9cc586c5b4c9cb501939c587d6a6e'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:15.598339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3effec91-c4c7-40c3-bead-c83545de7dd5 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:15.598339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3effec91-c4c7-40c3-bead-c83545de7dd5 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:15.598339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3effec91-c4c7-40c3-bead-c83545de7dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:15.614428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-b8f9caa7-b57a-4813-9f5a-eff8a1dfce80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bf9cc586c5b4c9cb501939c587d6a6e'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:15.625585 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-3effec91-c4c7-40c3-bead-c83545de7dd5 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:15.627426 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-3effec91-c4c7-40c3-bead-c83545de7dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:15.631395 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-b8f9caa7-b57a-4813-9f5a-eff8a1dfce80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bf9cc586c5b4c9cb501939c587d6a6e'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:15.639203 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 887/1772] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:15 2022] DELETE /identity/v3/users/1bf9cc586c5b4c9cb501939c587d6a6e => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:29:15.643754 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4273b15-a837-4a5e-aa7f-7c2c2db3a66d None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:15.644284 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4273b15-a837-4a5e-aa7f-7c2c2db3a66d None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:15.644732 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4273b15-a837-4a5e-aa7f-7c2c2db3a66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:15.645671 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 886/1773] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:15.649628 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3effec91-c4c7-40c3-bead-c83545de7dd5 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:15.666683 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3effec91-c4c7-40c3-bead-c83545de7dd5 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:15.667342 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-c4273b15-a837-4a5e-aa7f-7c2c2db3a66d None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:15.668646 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-c4273b15-a837-4a5e-aa7f-7c2c2db3a66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:15.743491 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 888/1774] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:15.753861 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4273b15-a837-4a5e-aa7f-7c2c2db3a66d None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:15.755663 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-c4273b15-a837-4a5e-aa7f-7c2c2db3a66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:15.758005 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1d965-10bd-4f79-a275-04ca72aa7905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:15.759475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1d965-10bd-4f79-a275-04ca72aa7905 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:15.759752 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1d965-10bd-4f79-a275-04ca72aa7905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:15.760077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1d965-10bd-4f79-a275-04ca72aa7905 admin admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:15.760676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f1d965-10bd-4f79-a275-04ca72aa7905 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:15.760981 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f1d965-10bd-4f79-a275-04ca72aa7905 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:15.762681 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f1d965-10bd-4f79-a275-04ca72aa7905 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:15.798859 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 887/1775] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:29:15.799532 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1d965-10bd-4f79-a275-04ca72aa7905 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:15.811026 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1d965-10bd-4f79-a275-04ca72aa7905 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:15.818922 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be038d2-011a-4668-9e7e-a4087554e355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:15.819561 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be038d2-011a-4668-9e7e-a4087554e355 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:15.819842 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be038d2-011a-4668-9e7e-a4087554e355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:15.820112 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be038d2-011a-4668-9e7e-a4087554e355 admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:15.820747 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be038d2-011a-4668-9e7e-a4087554e355 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:15.822648 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be038d2-011a-4668-9e7e-a4087554e355 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:15.842854 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4273b15-a837-4a5e-aa7f-7c2c2db3a66d None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:15.847615 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251323ef-17c6-4d7b-8c72-b7d1b0353161 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:15.848304 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251323ef-17c6-4d7b-8c72-b7d1b0353161 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:15.848759 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251323ef-17c6-4d7b-8c72-b7d1b0353161 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:15.848953 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251323ef-17c6-4d7b-8c72-b7d1b0353161 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:15.860938 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 888/1776] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:15 2022] POST /identity/v3/projects => generated 361 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:29:15.862922 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be038d2-011a-4668-9e7e-a4087554e355 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:15.870927 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251323ef-17c6-4d7b-8c72-b7d1b0353161 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:15.872404 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251323ef-17c6-4d7b-8c72-b7d1b0353161 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:15.893451 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:15.897287 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 889/1777] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:29:15.928750 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be038d2-011a-4668-9e7e-a4087554e355 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:15.933610 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aaa367-4c77-48e7-a87d-654ff9a73cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:15.934223 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aaa367-4c77-48e7-a87d-654ff9a73cd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:15.934494 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aaa367-4c77-48e7-a87d-654ff9a73cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:15.934788 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aaa367-4c77-48e7-a87d-654ff9a73cd6 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:15.935528 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aaa367-4c77-48e7-a87d-654ff9a73cd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:15.937662 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aaa367-4c77-48e7-a87d-654ff9a73cd6 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:15.961697 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 889/1778] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:15 2022] POST /identity/v3/users => generated 281 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:29:15.966418 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251323ef-17c6-4d7b-8c72-b7d1b0353161 service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:15.976987 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251323ef-17c6-4d7b-8c72-b7d1b0353161 service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:15.981643 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0a2aa-0b14-443b-b5f7-98f5ed707426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:15.982242 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0a2aa-0b14-443b-b5f7-98f5ed707426 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:15.982514 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0a2aa-0b14-443b-b5f7-98f5ed707426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:15.982794 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0a2aa-0b14-443b-b5f7-98f5ed707426 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:15.983428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d0a2aa-0b14-443b-b5f7-98f5ed707426 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:15.985436 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d0a2aa-0b14-443b-b5f7-98f5ed707426 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:15.996000 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 890/1779] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:16.000499 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aaa367-4c77-48e7-a87d-654ff9a73cd6 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:16.011966 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aaa367-4c77-48e7-a87d-654ff9a73cd6 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:16.017253 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89380a0e-a251-4f5c-bb5d-b9e75751eb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:16.017943 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89380a0e-a251-4f5c-bb5d-b9e75751eb0a admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:16.018402 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89380a0e-a251-4f5c-bb5d-b9e75751eb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:16.018792 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89380a0e-a251-4f5c-bb5d-b9e75751eb0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:16.019508 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89380a0e-a251-4f5c-bb5d-b9e75751eb0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:16.021187 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89380a0e-a251-4f5c-bb5d-b9e75751eb0a admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:16.035667 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 890/1780] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:29:16.036398 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0a2aa-0b14-443b-b5f7-98f5ed707426 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:16.052834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0a2aa-0b14-443b-b5f7-98f5ed707426 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:16.057617 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09622378-fe32-4e29-a523-d2b29901aec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:16.058315 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09622378-fe32-4e29-a523-d2b29901aec0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:16.058594 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09622378-fe32-4e29-a523-d2b29901aec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:16.059010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09622378-fe32-4e29-a523-d2b29901aec0 admin admin] PATH_INFO: `/v3/projects/1a777a9301814ac8bc819fa8066e1b29/users/7512017c890f4fb897d18a0196ae363c/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:16.077282 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09622378-fe32-4e29-a523-d2b29901aec0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a777a9301814ac8bc819fa8066e1b29, user_id=7512017c890f4fb897d18a0196ae363c, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:16.080564 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09622378-fe32-4e29-a523-d2b29901aec0 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:16.105469 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 891/1781] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:16 2022] PUT /identity/v3/projects/1a777a9301814ac8bc819fa8066e1b29/users/7512017c890f4fb897d18a0196ae363c/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:16.110452 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63840ae3-080d-40d3-80f7-843d92f18667 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:16.110452 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63840ae3-080d-40d3-80f7-843d92f18667 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:16.110452 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63840ae3-080d-40d3-80f7-843d92f18667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:16.137986 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-63840ae3-080d-40d3-80f7-843d92f18667 None None] MFA Rules not processed for user `7512017c890f4fb897d18a0196ae363c`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:16.140074 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-63840ae3-080d-40d3-80f7-843d92f18667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:16.214740 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 891/1782] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:16.252139 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09622378-fe32-4e29-a523-d2b29901aec0 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:16.254229 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-09622378-fe32-4e29-a523-d2b29901aec0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:16.293285 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:16.319716 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09622378-fe32-4e29-a523-d2b29901aec0 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:16.324161 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d443b7-8e79-481b-a759-274e568921ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:16.324983 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d443b7-8e79-481b-a759-274e568921ac service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:16.325252 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d443b7-8e79-481b-a759-274e568921ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:16.325585 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d443b7-8e79-481b-a759-274e568921ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:16.335499 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d443b7-8e79-481b-a759-274e568921ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:16.337003 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d443b7-8e79-481b-a759-274e568921ac service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:16.346051 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 892/1783] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:16.362607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63840ae3-080d-40d3-80f7-843d92f18667 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:16.381769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d443b7-8e79-481b-a759-274e568921ac service cinder] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:16.383811 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-14d443b7-8e79-481b-a759-274e568921ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:16.440039 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63840ae3-080d-40d3-80f7-843d92f18667 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:16.444538 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7befc296-541e-4156-80b1-d4e61397dff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:16.445179 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7befc296-541e-4156-80b1-d4e61397dff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:16.445450 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7befc296-541e-4156-80b1-d4e61397dff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:16.445715 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7befc296-541e-4156-80b1-d4e61397dff4 admin admin] PATH_INFO: `/v3/users/7512017c890f4fb897d18a0196ae363c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:16.446896 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7befc296-541e-4156-80b1-d4e61397dff4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7512017c890f4fb897d18a0196ae363c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:16.448836 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7befc296-541e-4156-80b1-d4e61397dff4 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:16.459597 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d443b7-8e79-481b-a759-274e568921ac service cinder] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:16.464272 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629884ad-948c-40b5-99c2-894eb9d53147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:16.464944 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629884ad-948c-40b5-99c2-894eb9d53147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:16.465218 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629884ad-948c-40b5-99c2-894eb9d53147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:16.465488 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629884ad-948c-40b5-99c2-894eb9d53147 admin admin] PATH_INFO: `/v3/projects/21c827bae43347d4b3537d5036df9548` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:16.466605 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629884ad-948c-40b5-99c2-894eb9d53147 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21c827bae43347d4b3537d5036df9548)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:16.468789 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629884ad-948c-40b5-99c2-894eb9d53147 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:16.490415 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:16.500321 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-7befc296-541e-4156-80b1-d4e61397dff4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7512017c890f4fb897d18a0196ae363c'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:16.502206 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-7befc296-541e-4156-80b1-d4e61397dff4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7512017c890f4fb897d18a0196ae363c'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:16.510294 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-7befc296-541e-4156-80b1-d4e61397dff4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7512017c890f4fb897d18a0196ae363c'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:16.520313 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-629884ad-948c-40b5-99c2-894eb9d53147 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21c827bae43347d4b3537d5036df9548', 'request_id': 'req-629884ad-948c-40b5-99c2-894eb9d53147'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:16.522624 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-7befc296-541e-4156-80b1-d4e61397dff4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7512017c890f4fb897d18a0196ae363c'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:16.529396 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-629884ad-948c-40b5-99c2-894eb9d53147 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21c827bae43347d4b3537d5036df9548', 'request_id': 'req-629884ad-948c-40b5-99c2-894eb9d53147'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:16.533214 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-7befc296-541e-4156-80b1-d4e61397dff4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7512017c890f4fb897d18a0196ae363c'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:16.536922 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-629884ad-948c-40b5-99c2-894eb9d53147 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21c827bae43347d4b3537d5036df9548', 'request_id': 'req-629884ad-948c-40b5-99c2-894eb9d53147'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:16.540376 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 892/1784] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:16 2022] DELETE /identity/v3/users/7512017c890f4fb897d18a0196ae363c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:16.545248 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7406f-c42c-4023-be82-5ac2c1e8a4a4 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:16.545466 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7406f-c42c-4023-be82-5ac2c1e8a4a4 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:16.545895 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7406f-c42c-4023-be82-5ac2c1e8a4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:16.550006 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-629884ad-948c-40b5-99c2-894eb9d53147 admin admin] The token cache is being invalidate because project 21c827bae43347d4b3537d5036df9548 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:16.550300 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-629884ad-948c-40b5-99c2-894eb9d53147 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:16.551006 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-629884ad-948c-40b5-99c2-894eb9d53147 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:16.552537 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 893/1785] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:16 2022] DELETE /identity/v3/projects/21c827bae43347d4b3537d5036df9548 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:29:16.556355 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629884ad-948c-40b5-99c2-894eb9d53147 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:16.557736 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-629884ad-948c-40b5-99c2-894eb9d53147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:16.567643 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-89a7406f-c42c-4023-be82-5ac2c1e8a4a4 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:16.569507 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-89a7406f-c42c-4023-be82-5ac2c1e8a4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:16.633746 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629884ad-948c-40b5-99c2-894eb9d53147 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:16.643098 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:16.661915 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 893/1786] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:16.672208 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a7406f-c42c-4023-be82-5ac2c1e8a4a4 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:16.673586 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-89a7406f-c42c-4023-be82-5ac2c1e8a4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:16.692486 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9dcaaa-4be5-40cf-9f0d-0f4e12164a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:16.693132 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9dcaaa-4be5-40cf-9f0d-0f4e12164a9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:16.693407 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9dcaaa-4be5-40cf-9f0d-0f4e12164a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:16.693678 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9dcaaa-4be5-40cf-9f0d-0f4e12164a9d admin admin] PATH_INFO: `/v3/users/bd73809ca24242d4afbe8d963779bc9a` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:16.694840 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9dcaaa-4be5-40cf-9f0d-0f4e12164a9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd73809ca24242d4afbe8d963779bc9a)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:16.696773 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9dcaaa-4be5-40cf-9f0d-0f4e12164a9d admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:16.731557 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a7406f-c42c-4023-be82-5ac2c1e8a4a4 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:16.737054 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2f3a9-5653-4461-b008-30bcf026441a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:16.737679 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2f3a9-5653-4461-b008-30bcf026441a service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:16.737956 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2f3a9-5653-4461-b008-30bcf026441a service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:16.738254 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2f3a9-5653-4461-b008-30bcf026441a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:16.747956 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-6b9dcaaa-4be5-40cf-9f0d-0f4e12164a9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd73809ca24242d4afbe8d963779bc9a'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:16.748267 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a2f3a9-5653-4461-b008-30bcf026441a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:16.749735 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a2f3a9-5653-4461-b008-30bcf026441a service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:16.755679 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-6b9dcaaa-4be5-40cf-9f0d-0f4e12164a9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd73809ca24242d4afbe8d963779bc9a'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:16.756382 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-6b9dcaaa-4be5-40cf-9f0d-0f4e12164a9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd73809ca24242d4afbe8d963779bc9a'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:16.760297 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 894/1787] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:29:16.768928 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-6b9dcaaa-4be5-40cf-9f0d-0f4e12164a9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd73809ca24242d4afbe8d963779bc9a'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:16.777261 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-6b9dcaaa-4be5-40cf-9f0d-0f4e12164a9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd73809ca24242d4afbe8d963779bc9a'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:16.787351 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 894/1788] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:16 2022] DELETE /identity/v3/users/bd73809ca24242d4afbe8d963779bc9a => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:29:17.184213 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2f3a9-5653-4461-b008-30bcf026441a service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:17.186214 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-23a2f3a9-5653-4461-b008-30bcf026441a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:17.201344 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2f3a9-5653-4461-b008-30bcf026441a service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:17.262535 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:17.270058 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e088be8-b04f-4677-8ecc-d05c67c8a009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:17.270898 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e088be8-b04f-4677-8ecc-d05c67c8a009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:17.271179 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e088be8-b04f-4677-8ecc-d05c67c8a009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:17.271452 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e088be8-b04f-4677-8ecc-d05c67c8a009 admin admin] PATH_INFO: `/v3/projects/1a777a9301814ac8bc819fa8066e1b29` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:17.272953 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9dcaaa-4be5-40cf-9f0d-0f4e12164a9d admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:17.274118 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e088be8-b04f-4677-8ecc-d05c67c8a009 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a777a9301814ac8bc819fa8066e1b29)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:17.274792 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-6b9dcaaa-4be5-40cf-9f0d-0f4e12164a9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:17.276167 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e088be8-b04f-4677-8ecc-d05c67c8a009 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:17.290382 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9dcaaa-4be5-40cf-9f0d-0f4e12164a9d admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:17.295315 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754c869-65ef-440e-9e2f-49a9420e0fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:17.296034 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754c869-65ef-440e-9e2f-49a9420e0fce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:17.296534 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754c869-65ef-440e-9e2f-49a9420e0fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:17.297128 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754c869-65ef-440e-9e2f-49a9420e0fce admin admin] PATH_INFO: `/v3/projects/93eb23200d80433ebfb35271045637b0` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:17.298530 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1754c869-65ef-440e-9e2f-49a9420e0fce admin admin] RBAC: Authorizing `identity:delete_project(project_id=93eb23200d80433ebfb35271045637b0)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:17.300728 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1754c869-65ef-440e-9e2f-49a9420e0fce admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:17.329266 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-3e088be8-b04f-4677-8ecc-d05c67c8a009 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a777a9301814ac8bc819fa8066e1b29', 'request_id': 'req-3e088be8-b04f-4677-8ecc-d05c67c8a009'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:17.336493 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-3e088be8-b04f-4677-8ecc-d05c67c8a009 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a777a9301814ac8bc819fa8066e1b29', 'request_id': 'req-3e088be8-b04f-4677-8ecc-d05c67c8a009'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:17.342922 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-3e088be8-b04f-4677-8ecc-d05c67c8a009 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a777a9301814ac8bc819fa8066e1b29', 'request_id': 'req-3e088be8-b04f-4677-8ecc-d05c67c8a009'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:17.353523 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-1754c869-65ef-440e-9e2f-49a9420e0fce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93eb23200d80433ebfb35271045637b0', 'request_id': 'req-1754c869-65ef-440e-9e2f-49a9420e0fce'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:17.356013 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-3e088be8-b04f-4677-8ecc-d05c67c8a009 admin admin] The token cache is being invalidate because project 1a777a9301814ac8bc819fa8066e1b29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:17.356367 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-3e088be8-b04f-4677-8ecc-d05c67c8a009 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:17.357038 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-3e088be8-b04f-4677-8ecc-d05c67c8a009 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:17.358875 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 895/1789] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:17 2022] DELETE /identity/v3/projects/1a777a9301814ac8bc819fa8066e1b29 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:17.363851 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-1754c869-65ef-440e-9e2f-49a9420e0fce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93eb23200d80433ebfb35271045637b0', 'request_id': 'req-1754c869-65ef-440e-9e2f-49a9420e0fce'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:17.364162 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de720b13-5a85-4d7d-be22-43a3cbc598da None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:17.364444 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de720b13-5a85-4d7d-be22-43a3cbc598da None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:17.364703 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de720b13-5a85-4d7d-be22-43a3cbc598da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:17.369802 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-1754c869-65ef-440e-9e2f-49a9420e0fce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93eb23200d80433ebfb35271045637b0', 'request_id': 'req-1754c869-65ef-440e-9e2f-49a9420e0fce'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:17.382091 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-1754c869-65ef-440e-9e2f-49a9420e0fce admin admin] The token cache is being invalidate because project 93eb23200d80433ebfb35271045637b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:17.382301 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-1754c869-65ef-440e-9e2f-49a9420e0fce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:17.382951 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-1754c869-65ef-440e-9e2f-49a9420e0fce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:17.384483 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 895/1790] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:17 2022] DELETE /identity/v3/projects/93eb23200d80433ebfb35271045637b0 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:29:17.389170 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-de720b13-5a85-4d7d-be22-43a3cbc598da None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:17.389170 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-de720b13-5a85-4d7d-be22-43a3cbc598da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:17.391008 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd86f7-1572-4370-ad6f-b35a148c6af3 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:17.391314 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd86f7-1572-4370-ad6f-b35a148c6af3 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:17.391580 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd86f7-1572-4370-ad6f-b35a148c6af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:17.412019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-63cd86f7-1572-4370-ad6f-b35a148c6af3 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:17.413625 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-63cd86f7-1572-4370-ad6f-b35a148c6af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:17.423857 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:17.448163 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 896/1791] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:17.456321 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958d987-6a04-48cb-afef-76c803012210 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:17.456602 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958d987-6a04-48cb-afef-76c803012210 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:17.456871 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958d987-6a04-48cb-afef-76c803012210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:17.476329 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-f958d987-6a04-48cb-afef-76c803012210 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:17.477519 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-f958d987-6a04-48cb-afef-76c803012210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:17.479343 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 896/1792] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:17.486088 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 897/1793] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:17.488892 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a0e7c-fc3e-4c68-86a0-ec92d67d5d35 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:17.489369 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a0e7c-fc3e-4c68-86a0-ec92d67d5d35 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:17.489648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a0e7c-fc3e-4c68-86a0-ec92d67d5d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:17.490155 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f958d987-6a04-48cb-afef-76c803012210 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:17.499692 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f958d987-6a04-48cb-afef-76c803012210 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:17.504777 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fbd0c6-d00f-4729-b236-32c12a86af86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:17.505367 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbd0c6-d00f-4729-b236-32c12a86af86 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:17.505775 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbd0c6-d00f-4729-b236-32c12a86af86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:17.506102 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbd0c6-d00f-4729-b236-32c12a86af86 admin admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:17.506766 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fbd0c6-d00f-4729-b236-32c12a86af86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:17.507093 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fbd0c6-d00f-4729-b236-32c12a86af86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:17.508690 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fbd0c6-d00f-4729-b236-32c12a86af86 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:17.509642 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-343a0e7c-fc3e-4c68-86a0-ec92d67d5d35 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:17.510874 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-343a0e7c-fc3e-4c68-86a0-ec92d67d5d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:17.518893 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 897/1794] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:17.521245 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 898/1795] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:29:17.523102 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a0e7c-fc3e-4c68-86a0-ec92d67d5d35 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:17.525147 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fbd0c6-d00f-4729-b236-32c12a86af86 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:17.532839 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a0e7c-fc3e-4c68-86a0-ec92d67d5d35 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:17.535416 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fbd0c6-d00f-4729-b236-32c12a86af86 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:17.537511 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb41bd-94c5-421b-831e-06be5304d1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:17.538110 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb41bd-94c5-421b-831e-06be5304d1ca admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:17.538386 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb41bd-94c5-421b-831e-06be5304d1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:17.538677 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb41bd-94c5-421b-831e-06be5304d1ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:17.539403 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bb41bd-94c5-421b-831e-06be5304d1ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:17.539709 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bb41bd-94c5-421b-831e-06be5304d1ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:17.540819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d0758-3457-4149-95a6-0671106c1fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:17.541309 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bb41bd-94c5-421b-831e-06be5304d1ca admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:17.541419 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d0758-3457-4149-95a6-0671106c1fed admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:17.541695 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d0758-3457-4149-95a6-0671106c1fed admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:17.541968 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d0758-3457-4149-95a6-0671106c1fed admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:17.542587 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d0758-3457-4149-95a6-0671106c1fed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:17.544496 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d0758-3457-4149-95a6-0671106c1fed admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:17.553597 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 898/1796] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:29:17.558070 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb41bd-94c5-421b-831e-06be5304d1ca admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:17.569107 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb41bd-94c5-421b-831e-06be5304d1ca admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:17.573923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4003d5a-52e8-4522-9c2a-5925bb1a86b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:17.574568 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4003d5a-52e8-4522-9c2a-5925bb1a86b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:17.578198 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4003d5a-52e8-4522-9c2a-5925bb1a86b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:17.578327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 899/1797] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:17 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:29:17.578411 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4003d5a-52e8-4522-9c2a-5925bb1a86b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:17.578411 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4003d5a-52e8-4522-9c2a-5925bb1a86b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:17.578411 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4003d5a-52e8-4522-9c2a-5925bb1a86b9 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:17.579344 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d0758-3457-4149-95a6-0671106c1fed admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:17.621736 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 899/1798] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:17 2022] POST /identity/v3/projects => generated 377 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:17.622773 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4003d5a-52e8-4522-9c2a-5925bb1a86b9 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:17.645774 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:17.674832 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d0758-3457-4149-95a6-0671106c1fed admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:17.706837 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4003d5a-52e8-4522-9c2a-5925bb1a86b9 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:17.711794 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a157e9-4b73-4320-bbef-3ca4e45d24da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:17.712405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a157e9-4b73-4320-bbef-3ca4e45d24da admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:17.712679 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a157e9-4b73-4320-bbef-3ca4e45d24da admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:17.712945 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a157e9-4b73-4320-bbef-3ca4e45d24da admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:17.713572 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a157e9-4b73-4320-bbef-3ca4e45d24da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:17.715565 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a157e9-4b73-4320-bbef-3ca4e45d24da admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:17.739342 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b32daf-2a48-4b21-a24f-393d24c2314b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:17.739992 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b32daf-2a48-4b21-a24f-393d24c2314b admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:17.740269 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b32daf-2a48-4b21-a24f-393d24c2314b admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:17.740536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b32daf-2a48-4b21-a24f-393d24c2314b admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:17.741168 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b32daf-2a48-4b21-a24f-393d24c2314b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:17.743122 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b32daf-2a48-4b21-a24f-393d24c2314b admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:17.743985 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 900/1799] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:17 2022] POST /identity/v3/users => generated 289 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:29:17.748227 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a157e9-4b73-4320-bbef-3ca4e45d24da admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:17.759402 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a157e9-4b73-4320-bbef-3ca4e45d24da admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:17.764097 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accec705-1fc1-4c38-ba4d-1c4eeec5b9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:17.764697 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accec705-1fc1-4c38-ba4d-1c4eeec5b9d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:17.764996 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accec705-1fc1-4c38-ba4d-1c4eeec5b9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:17.765264 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accec705-1fc1-4c38-ba4d-1c4eeec5b9d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:17.765897 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accec705-1fc1-4c38-ba4d-1c4eeec5b9d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:17.767524 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accec705-1fc1-4c38-ba4d-1c4eeec5b9d6 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:17.771805 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 900/1800] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:17 2022] POST /identity/v3/users => generated 284 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:29:17.772657 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b32daf-2a48-4b21-a24f-393d24c2314b admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:17.782718 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 901/1801] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:29:17.782869 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b32daf-2a48-4b21-a24f-393d24c2314b admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:17.783686 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accec705-1fc1-4c38-ba4d-1c4eeec5b9d6 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:17.787183 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc22d7-5dd8-457a-bbe6-99c5d84a8b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:17.787778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc22d7-5dd8-457a-bbe6-99c5d84a8b5b admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:17.788051 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc22d7-5dd8-457a-bbe6-99c5d84a8b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:17.788316 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc22d7-5dd8-457a-bbe6-99c5d84a8b5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:17.788969 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc22d7-5dd8-457a-bbe6-99c5d84a8b5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:17.790559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc22d7-5dd8-457a-bbe6-99c5d84a8b5b admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:17.792995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accec705-1fc1-4c38-ba4d-1c4eeec5b9d6 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:17.798016 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc70376-e6c8-4c67-86f9-d25e804e9f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:17.798606 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc70376-e6c8-4c67-86f9-d25e804e9f9c admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:17.798921 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc70376-e6c8-4c67-86f9-d25e804e9f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:17.799183 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc70376-e6c8-4c67-86f9-d25e804e9f9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:17.799808 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc70376-e6c8-4c67-86f9-d25e804e9f9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:17.801589 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc70376-e6c8-4c67-86f9-d25e804e9f9c admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:17.801701 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 901/1802] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:29:17.805305 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc22d7-5dd8-457a-bbe6-99c5d84a8b5b admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:17.816215 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 902/1803] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:29:17.816408 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc22d7-5dd8-457a-bbe6-99c5d84a8b5b admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:17.817177 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc70376-e6c8-4c67-86f9-d25e804e9f9c admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:17.820478 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d0619-0c10-477c-a032-daab89fd0b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:17.821084 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d0619-0c10-477c-a032-daab89fd0b65 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:17.821358 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d0619-0c10-477c-a032-daab89fd0b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:17.821627 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d0619-0c10-477c-a032-daab89fd0b65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:17.822254 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629d0619-0c10-477c-a032-daab89fd0b65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:17.823880 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629d0619-0c10-477c-a032-daab89fd0b65 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:17.826919 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc70376-e6c8-4c67-86f9-d25e804e9f9c admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:17.831551 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ab2b5-d6a4-48ec-a1c7-a131d5022afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:17.832184 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ab2b5-d6a4-48ec-a1c7-a131d5022afb admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:17.832454 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ab2b5-d6a4-48ec-a1c7-a131d5022afb admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:17.832715 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ab2b5-d6a4-48ec-a1c7-a131d5022afb admin admin] PATH_INFO: `/v3/projects/df987d9c213649969f9c92c14fb39d90/users/1063cf561f854909ac25c8d96b5009a4/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:17.834756 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 902/1804] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:29:17.838676 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d0619-0c10-477c-a032-daab89fd0b65 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:17.847776 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5ab2b5-d6a4-48ec-a1c7-a131d5022afb admin admin] RBAC: Authorizing `identity:create_grant(project_id=df987d9c213649969f9c92c14fb39d90, user_id=1063cf561f854909ac25c8d96b5009a4, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:17.848290 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d0619-0c10-477c-a032-daab89fd0b65 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:17.850593 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5ab2b5-d6a4-48ec-a1c7-a131d5022afb admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:17.852771 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa61363-b9e4-4905-ac40-0e2775df2300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:17.853413 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa61363-b9e4-4905-ac40-0e2775df2300 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:17.853691 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa61363-b9e4-4905-ac40-0e2775df2300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:17.853992 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa61363-b9e4-4905-ac40-0e2775df2300 admin admin] PATH_INFO: `/v3/projects/100f071407274b6aaeaaa8ed758df75f/users/23adbcf72828456daadee6a5dd1e5545/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:17.867112 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 903/1805] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:17 2022] PUT /identity/v3/projects/df987d9c213649969f9c92c14fb39d90/users/1063cf561f854909ac25c8d96b5009a4/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:17.870009 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa61363-b9e4-4905-ac40-0e2775df2300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=100f071407274b6aaeaaa8ed758df75f, user_id=23adbcf72828456daadee6a5dd1e5545, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:17.872694 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa61363-b9e4-4905-ac40-0e2775df2300 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:17.878956 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e57673-5b05-4eb7-bbf7-2bc3c282d47b None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:17.879255 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e57673-5b05-4eb7-bbf7-2bc3c282d47b None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:17.879548 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e57673-5b05-4eb7-bbf7-2bc3c282d47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:17.888049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 903/1806] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:17 2022] PUT /identity/v3/projects/100f071407274b6aaeaaa8ed758df75f/users/23adbcf72828456daadee6a5dd1e5545/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:29:17.899226 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb00e6-bf22-4352-ab52-715d1a42c2e0 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:17.899506 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb00e6-bf22-4352-ab52-715d1a42c2e0 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:17.899791 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb00e6-bf22-4352-ab52-715d1a42c2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:17.908835 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-a4e57673-5b05-4eb7-bbf7-2bc3c282d47b None None] MFA Rules not processed for user `1063cf561f854909ac25c8d96b5009a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:17.910058 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-a4e57673-5b05-4eb7-bbf7-2bc3c282d47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:17.915363 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:17.928272 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-adeb00e6-bf22-4352-ab52-715d1a42c2e0 None None] MFA Rules not processed for user `23adbcf72828456daadee6a5dd1e5545`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:17.929661 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-adeb00e6-bf22-4352-ab52-715d1a42c2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:17.976317 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 904/1807] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:17.988068 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e57673-5b05-4eb7-bbf7-2bc3c282d47b None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:17.989449 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-a4e57673-5b05-4eb7-bbf7-2bc3c282d47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:17.995992 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 904/1808] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:18.008549 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb00e6-bf22-4352-ab52-715d1a42c2e0 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:18.009986 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-adeb00e6-bf22-4352-ab52-715d1a42c2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:18.042131 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:18.081796 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb00e6-bf22-4352-ab52-715d1a42c2e0 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:18.084410 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e57673-5b05-4eb7-bbf7-2bc3c282d47b None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:18.085838 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8294bc-8629-476f-bd7d-05a66eaf5e40 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a051ca8154540ad93db6d028b8323a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:18.087175 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8294bc-8629-476f-bd7d-05a66eaf5e40 service glance] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:18.087175 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8294bc-8629-476f-bd7d-05a66eaf5e40 service glance] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:18.087175 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8294bc-8629-476f-bd7d-05a66eaf5e40 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:18.089606 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69bdf2c-89bc-4f6f-aacc-bd0d9696b351 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a051ca8154540ad93db6d028b8323a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:18.090164 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bdf2c-89bc-4f6f-aacc-bd0d9696b351 service glance] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:18.090442 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bdf2c-89bc-4f6f-aacc-bd0d9696b351 service glance] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:18.090744 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bdf2c-89bc-4f6f-aacc-bd0d9696b351 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:18.097466 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8294bc-8629-476f-bd7d-05a66eaf5e40 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:18.099111 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8294bc-8629-476f-bd7d-05a66eaf5e40 service glance] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:18.099820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69bdf2c-89bc-4f6f-aacc-bd0d9696b351 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:18.101303 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69bdf2c-89bc-4f6f-aacc-bd0d9696b351 service glance] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:18.108848 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 905/1809] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:29:18.116017 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 905/1810] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:20.698474 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8294bc-8629-476f-bd7d-05a66eaf5e40 service glance] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:20.703910 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-7f8294bc-8629-476f-bd7d-05a66eaf5e40 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:20.780522 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8294bc-8629-476f-bd7d-05a66eaf5e40 service glance] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:20.785604 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4947c-1f39-4744-895f-9ce009dbebfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:20.786209 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4947c-1f39-4744-895f-9ce009dbebfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:20.786486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4947c-1f39-4744-895f-9ce009dbebfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:20.786815 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4947c-1f39-4744-895f-9ce009dbebfc admin admin] PATH_INFO: `/v3/users/04940315f596486ebbf4bc1d824a5c42` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:20.787988 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4947c-1f39-4744-895f-9ce009dbebfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=04940315f596486ebbf4bc1d824a5c42)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:20.789908 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4947c-1f39-4744-895f-9ce009dbebfc admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:20.840884 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-95a4947c-1f39-4744-895f-9ce009dbebfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04940315f596486ebbf4bc1d824a5c42'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:20.841627 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-95a4947c-1f39-4744-895f-9ce009dbebfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04940315f596486ebbf4bc1d824a5c42'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:20.849917 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-95a4947c-1f39-4744-895f-9ce009dbebfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04940315f596486ebbf4bc1d824a5c42'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:20.868063 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-95a4947c-1f39-4744-895f-9ce009dbebfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04940315f596486ebbf4bc1d824a5c42'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:20.876950 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-95a4947c-1f39-4744-895f-9ce009dbebfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04940315f596486ebbf4bc1d824a5c42'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:20.882981 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 906/1811] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:20 2022] DELETE /identity/v3/users/04940315f596486ebbf4bc1d824a5c42 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:20.887743 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea6ac2-2bf5-45f7-b0ff-e3a7de19a483 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:20.888273 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea6ac2-2bf5-45f7-b0ff-e3a7de19a483 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:20.888273 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea6ac2-2bf5-45f7-b0ff-e3a7de19a483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:20.907783 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-7eea6ac2-2bf5-45f7-b0ff-e3a7de19a483 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:20.909062 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-7eea6ac2-2bf5-45f7-b0ff-e3a7de19a483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:20.975475 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 906/1812] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:20.986065 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4947c-1f39-4744-895f-9ce009dbebfc admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:20.987701 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-95a4947c-1f39-4744-895f-9ce009dbebfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:21.034516 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:21.050924 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4947c-1f39-4744-895f-9ce009dbebfc admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:21.055114 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134e0e04-a49e-47d7-a3fb-316572921df5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:21.055736 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e0e04-a49e-47d7-a3fb-316572921df5 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:21.056011 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e0e04-a49e-47d7-a3fb-316572921df5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:21.056281 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e0e04-a49e-47d7-a3fb-316572921df5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:21.066760 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134e0e04-a49e-47d7-a3fb-316572921df5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:21.068226 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134e0e04-a49e-47d7-a3fb-316572921df5 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:21.079576 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 907/1813] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:21.393018 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea6ac2-2bf5-45f7-b0ff-e3a7de19a483 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:21.397042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-7eea6ac2-2bf5-45f7-b0ff-e3a7de19a483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:21.408307 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea6ac2-2bf5-45f7-b0ff-e3a7de19a483 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:21.413307 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d17f0-21e8-4768-bd3b-0c4b33dafd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:21.414049 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d17f0-21e8-4768-bd3b-0c4b33dafd2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:21.414298 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d17f0-21e8-4768-bd3b-0c4b33dafd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:21.414946 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d17f0-21e8-4768-bd3b-0c4b33dafd2f admin admin] PATH_INFO: `/v3/projects/d2b4884c02874086acc5bf75e1da7ac4` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:21.415764 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d17f0-21e8-4768-bd3b-0c4b33dafd2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2b4884c02874086acc5bf75e1da7ac4)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:21.421249 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d17f0-21e8-4768-bd3b-0c4b33dafd2f admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:21.455674 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:21.478408 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0e8d17f0-21e8-4768-bd3b-0c4b33dafd2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2b4884c02874086acc5bf75e1da7ac4', 'request_id': 'req-0e8d17f0-21e8-4768-bd3b-0c4b33dafd2f'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:21.489393 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0e8d17f0-21e8-4768-bd3b-0c4b33dafd2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2b4884c02874086acc5bf75e1da7ac4', 'request_id': 'req-0e8d17f0-21e8-4768-bd3b-0c4b33dafd2f'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:21.505102 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0e8d17f0-21e8-4768-bd3b-0c4b33dafd2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2b4884c02874086acc5bf75e1da7ac4', 'request_id': 'req-0e8d17f0-21e8-4768-bd3b-0c4b33dafd2f'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:21.517482 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0e8d17f0-21e8-4768-bd3b-0c4b33dafd2f admin admin] The token cache is being invalidate because project d2b4884c02874086acc5bf75e1da7ac4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:21.517772 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0e8d17f0-21e8-4768-bd3b-0c4b33dafd2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:21.518449 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0e8d17f0-21e8-4768-bd3b-0c4b33dafd2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:21.520137 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 907/1814] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:21 2022] DELETE /identity/v3/projects/d2b4884c02874086acc5bf75e1da7ac4 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:21.526069 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372ac10-6d1e-41ed-973e-ce06f526ea49 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:21.526445 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372ac10-6d1e-41ed-973e-ce06f526ea49 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:21.526755 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372ac10-6d1e-41ed-973e-ce06f526ea49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:21.548122 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-b372ac10-6d1e-41ed-973e-ce06f526ea49 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:21.549494 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-b372ac10-6d1e-41ed-973e-ce06f526ea49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:21.622528 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 908/1815] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:21.631321 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d97c5-ccf9-4493-9796-96ef2885d5a1 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:21.631604 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d97c5-ccf9-4493-9796-96ef2885d5a1 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:21.631874 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d97c5-ccf9-4493-9796-96ef2885d5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:21.659377 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-803d97c5-ccf9-4493-9796-96ef2885d5a1 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:21.660626 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-803d97c5-ccf9-4493-9796-96ef2885d5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:21.669923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 908/1816] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:21.675363 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372ac10-6d1e-41ed-973e-ce06f526ea49 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:21.686477 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372ac10-6d1e-41ed-973e-ce06f526ea49 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:21.692014 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f7343-0f15-4b09-bd8f-57954017534e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:21.692632 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f7343-0f15-4b09-bd8f-57954017534e admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:21.692907 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f7343-0f15-4b09-bd8f-57954017534e admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:21.693169 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f7343-0f15-4b09-bd8f-57954017534e admin admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:21.693815 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f7343-0f15-4b09-bd8f-57954017534e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:21.694108 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f7343-0f15-4b09-bd8f-57954017534e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:21.695768 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f7343-0f15-4b09-bd8f-57954017534e admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:21.708266 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 909/1817] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:29:21.713153 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d97c5-ccf9-4493-9796-96ef2885d5a1 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:21.723072 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d97c5-ccf9-4493-9796-96ef2885d5a1 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:21.728185 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b312b4-24bd-471a-b219-f5415bea3440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:21.728811 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b312b4-24bd-471a-b219-f5415bea3440 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:21.729086 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b312b4-24bd-471a-b219-f5415bea3440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:21.729356 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b312b4-24bd-471a-b219-f5415bea3440 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:21.730016 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b312b4-24bd-471a-b219-f5415bea3440 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:21.732259 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b312b4-24bd-471a-b219-f5415bea3440 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:21.765634 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 909/1818] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:21 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:29:21.769869 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f7343-0f15-4b09-bd8f-57954017534e admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:21.785784 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f7343-0f15-4b09-bd8f-57954017534e admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:21.865074 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b42f0f-7f02-434e-802c-4eeb383e75da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:21.865797 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b42f0f-7f02-434e-802c-4eeb383e75da admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:21.866140 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b42f0f-7f02-434e-802c-4eeb383e75da admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:21.866477 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b42f0f-7f02-434e-802c-4eeb383e75da admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:21.867300 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b42f0f-7f02-434e-802c-4eeb383e75da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:21.869311 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b42f0f-7f02-434e-802c-4eeb383e75da admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:21.897655 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 910/1819] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:21 2022] POST /identity/v3/users => generated 292 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:21.901308 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b312b4-24bd-471a-b219-f5415bea3440 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:21.913570 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b312b4-24bd-471a-b219-f5415bea3440 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:21.918821 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2d774-1807-4690-b56e-f45acdcab29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:21.919679 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2d774-1807-4690-b56e-f45acdcab29d admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:21.920023 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2d774-1807-4690-b56e-f45acdcab29d admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:21.920552 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2d774-1807-4690-b56e-f45acdcab29d admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:21.921322 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff2d774-1807-4690-b56e-f45acdcab29d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:21.923525 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff2d774-1807-4690-b56e-f45acdcab29d admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:21.934331 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 910/1820] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:21.941738 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b42f0f-7f02-434e-802c-4eeb383e75da admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:21.952598 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b42f0f-7f02-434e-802c-4eeb383e75da admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:21.957683 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d635198-2432-459e-9f14-431528344d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:21.958229 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d635198-2432-459e-9f14-431528344d2f admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:21.958471 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d635198-2432-459e-9f14-431528344d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:21.958776 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d635198-2432-459e-9f14-431528344d2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:21.959495 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d635198-2432-459e-9f14-431528344d2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:21.961112 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d635198-2432-459e-9f14-431528344d2f admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:21.973659 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 911/1821] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:29:21.977863 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2d774-1807-4690-b56e-f45acdcab29d admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:22.006754 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2d774-1807-4690-b56e-f45acdcab29d admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:22.012467 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95518e-27cf-4ee7-ada3-c91a9087facb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:22.013153 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95518e-27cf-4ee7-ada3-c91a9087facb admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:22.013481 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95518e-27cf-4ee7-ada3-c91a9087facb admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:22.013758 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95518e-27cf-4ee7-ada3-c91a9087facb admin admin] PATH_INFO: `/v3/projects/4866e41079fb4b0298212f80dc15399c/users/25afd7d8fcfe4dcf9133872d69520508/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:22.033575 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b95518e-27cf-4ee7-ada3-c91a9087facb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4866e41079fb4b0298212f80dc15399c, user_id=25afd7d8fcfe4dcf9133872d69520508, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:22.036810 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b95518e-27cf-4ee7-ada3-c91a9087facb admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:22.051083 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 911/1822] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:21 2022] PUT /identity/v3/projects/4866e41079fb4b0298212f80dc15399c/users/25afd7d8fcfe4dcf9133872d69520508/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:22.056422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b4727-b3a6-4a72-a3f3-9d9eb2314806 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:22.056701 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b4727-b3a6-4a72-a3f3-9d9eb2314806 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:22.056969 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b4727-b3a6-4a72-a3f3-9d9eb2314806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:22.077969 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-b95b4727-b3a6-4a72-a3f3-9d9eb2314806 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:22.079209 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-b95b4727-b3a6-4a72-a3f3-9d9eb2314806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:22.132487 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:22.153645 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 912/1823] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:22.161268 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95518e-27cf-4ee7-ada3-c91a9087facb admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:22.163686 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-3b95518e-27cf-4ee7-ada3-c91a9087facb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:22.230772 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95518e-27cf-4ee7-ada3-c91a9087facb admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:22.236044 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8057a0-c3af-417a-a8a4-1d70c9e49f73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:22.236848 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8057a0-c3af-417a-a8a4-1d70c9e49f73 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:22.237151 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8057a0-c3af-417a-a8a4-1d70c9e49f73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:22.237299 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8057a0-c3af-417a-a8a4-1d70c9e49f73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:22.247882 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8057a0-c3af-417a-a8a4-1d70c9e49f73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:22.249318 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8057a0-c3af-417a-a8a4-1d70c9e49f73 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:22.260526 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 912/1824] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:23.057134 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b4727-b3a6-4a72-a3f3-9d9eb2314806 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:23.058700 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-b95b4727-b3a6-4a72-a3f3-9d9eb2314806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:23.072585 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b4727-b3a6-4a72-a3f3-9d9eb2314806 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:23.078106 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4560765-10c1-4b30-b97b-09e9719b2869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:23.079272 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4560765-10c1-4b30-b97b-09e9719b2869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:23.079816 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4560765-10c1-4b30-b97b-09e9719b2869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:23.080262 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4560765-10c1-4b30-b97b-09e9719b2869 admin admin] PATH_INFO: `/v3/users/dbcbff89b82d41a1b0188eff9f569144` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:23.081978 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4560765-10c1-4b30-b97b-09e9719b2869 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbcbff89b82d41a1b0188eff9f569144)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:23.085054 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4560765-10c1-4b30-b97b-09e9719b2869 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:23.146310 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-e4560765-10c1-4b30-b97b-09e9719b2869 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbcbff89b82d41a1b0188eff9f569144'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:23.146997 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-e4560765-10c1-4b30-b97b-09e9719b2869 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbcbff89b82d41a1b0188eff9f569144'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:23.154437 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-e4560765-10c1-4b30-b97b-09e9719b2869 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbcbff89b82d41a1b0188eff9f569144'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:23.174359 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-e4560765-10c1-4b30-b97b-09e9719b2869 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbcbff89b82d41a1b0188eff9f569144'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:23.184505 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-e4560765-10c1-4b30-b97b-09e9719b2869 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbcbff89b82d41a1b0188eff9f569144'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:23.193431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 913/1825] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:23 2022] DELETE /identity/v3/users/dbcbff89b82d41a1b0188eff9f569144 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:23.198513 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744bcdb1-d56e-4541-95be-8b6039c7da63 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:23.199107 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744bcdb1-d56e-4541-95be-8b6039c7da63 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:23.199107 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744bcdb1-d56e-4541-95be-8b6039c7da63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:23.219697 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-744bcdb1-d56e-4541-95be-8b6039c7da63 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:23.221075 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-744bcdb1-d56e-4541-95be-8b6039c7da63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:23.277267 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:23.294473 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 913/1826] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:23.304757 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4560765-10c1-4b30-b97b-09e9719b2869 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:23.306243 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-e4560765-10c1-4b30-b97b-09e9719b2869 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:23.389034 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4560765-10c1-4b30-b97b-09e9719b2869 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:23.394771 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043023cf-79ea-4042-b841-5c0f0d5fbead service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:23.395497 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043023cf-79ea-4042-b841-5c0f0d5fbead service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:23.395828 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043023cf-79ea-4042-b841-5c0f0d5fbead service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:23.396180 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043023cf-79ea-4042-b841-5c0f0d5fbead service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:23.406038 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043023cf-79ea-4042-b841-5c0f0d5fbead service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:23.407862 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043023cf-79ea-4042-b841-5c0f0d5fbead service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:23.422973 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 914/1827] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:23.759749 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744bcdb1-d56e-4541-95be-8b6039c7da63 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:23.762537 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-744bcdb1-d56e-4541-95be-8b6039c7da63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:23.787721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744bcdb1-d56e-4541-95be-8b6039c7da63 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:23.794271 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552092fc-8e7c-4acb-b078-4a23d485a357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:23.795371 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552092fc-8e7c-4acb-b078-4a23d485a357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:23.795663 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552092fc-8e7c-4acb-b078-4a23d485a357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:23.795938 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552092fc-8e7c-4acb-b078-4a23d485a357 admin admin] PATH_INFO: `/v3/projects/71bf346a33bc4828bf5750e94ee1cbeb` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:23.797805 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552092fc-8e7c-4acb-b078-4a23d485a357 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71bf346a33bc4828bf5750e94ee1cbeb)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:23.800275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552092fc-8e7c-4acb-b078-4a23d485a357 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:23.864667 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-552092fc-8e7c-4acb-b078-4a23d485a357 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71bf346a33bc4828bf5750e94ee1cbeb', 'request_id': 'req-552092fc-8e7c-4acb-b078-4a23d485a357'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:23.881637 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-552092fc-8e7c-4acb-b078-4a23d485a357 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71bf346a33bc4828bf5750e94ee1cbeb', 'request_id': 'req-552092fc-8e7c-4acb-b078-4a23d485a357'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:23.889535 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-552092fc-8e7c-4acb-b078-4a23d485a357 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71bf346a33bc4828bf5750e94ee1cbeb', 'request_id': 'req-552092fc-8e7c-4acb-b078-4a23d485a357'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:23.905267 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-552092fc-8e7c-4acb-b078-4a23d485a357 admin admin] The token cache is being invalidate because project 71bf346a33bc4828bf5750e94ee1cbeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:23.905601 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-552092fc-8e7c-4acb-b078-4a23d485a357 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:23.906520 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-552092fc-8e7c-4acb-b078-4a23d485a357 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:23.911748 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 914/1828] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:23 2022] DELETE /identity/v3/projects/71bf346a33bc4828bf5750e94ee1cbeb => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:23.912567 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043023cf-79ea-4042-b841-5c0f0d5fbead service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:23.924509 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043023cf-79ea-4042-b841-5c0f0d5fbead service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:24.043202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbbbe89-c4d7-4782-91d8-764a071e6e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:24.044871 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbbbe89-c4d7-4782-91d8-764a071e6e50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:24.045244 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbbbe89-c4d7-4782-91d8-764a071e6e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:24.046129 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbbbe89-c4d7-4782-91d8-764a071e6e50 admin admin] PATH_INFO: `/v3/users/774fe1201baa4a0b922a35dacb419f38` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:24.058734 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbbbe89-c4d7-4782-91d8-764a071e6e50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=774fe1201baa4a0b922a35dacb419f38)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:24.060700 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbbbe89-c4d7-4782-91d8-764a071e6e50 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:24.079147 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:24.160193 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-cfbbbe89-c4d7-4782-91d8-764a071e6e50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '774fe1201baa4a0b922a35dacb419f38'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:24.161379 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-cfbbbe89-c4d7-4782-91d8-764a071e6e50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '774fe1201baa4a0b922a35dacb419f38'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:24.171041 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-cfbbbe89-c4d7-4782-91d8-764a071e6e50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '774fe1201baa4a0b922a35dacb419f38'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:24.186957 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-cfbbbe89-c4d7-4782-91d8-764a071e6e50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '774fe1201baa4a0b922a35dacb419f38'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:24.201160 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-cfbbbe89-c4d7-4782-91d8-764a071e6e50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '774fe1201baa4a0b922a35dacb419f38'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:24.210697 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 915/1829] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:23 2022] DELETE /identity/v3/users/774fe1201baa4a0b922a35dacb419f38 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:29:24.667708 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552092fc-8e7c-4acb-b078-4a23d485a357 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:24.669490 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-552092fc-8e7c-4acb-b078-4a23d485a357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:24.691663 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552092fc-8e7c-4acb-b078-4a23d485a357 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:24.797165 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce66b09-58e9-4d98-9d16-bde1c08b84dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:24.798061 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce66b09-58e9-4d98-9d16-bde1c08b84dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:24.798512 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce66b09-58e9-4d98-9d16-bde1c08b84dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:24.798759 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce66b09-58e9-4d98-9d16-bde1c08b84dd admin admin] PATH_INFO: `/v3/projects/7da9159cbb5448fb9361edd9e92f2713` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:24.800019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce66b09-58e9-4d98-9d16-bde1c08b84dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7da9159cbb5448fb9361edd9e92f2713)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:24.802779 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce66b09-58e9-4d98-9d16-bde1c08b84dd admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:24.924196 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5ce66b09-58e9-4d98-9d16-bde1c08b84dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7da9159cbb5448fb9361edd9e92f2713', 'request_id': 'req-5ce66b09-58e9-4d98-9d16-bde1c08b84dd'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:24.950508 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5ce66b09-58e9-4d98-9d16-bde1c08b84dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7da9159cbb5448fb9361edd9e92f2713', 'request_id': 'req-5ce66b09-58e9-4d98-9d16-bde1c08b84dd'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:24.965749 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5ce66b09-58e9-4d98-9d16-bde1c08b84dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7da9159cbb5448fb9361edd9e92f2713', 'request_id': 'req-5ce66b09-58e9-4d98-9d16-bde1c08b84dd'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:24.996796 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5ce66b09-58e9-4d98-9d16-bde1c08b84dd admin admin] The token cache is being invalidate because project 7da9159cbb5448fb9361edd9e92f2713 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:24.997385 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5ce66b09-58e9-4d98-9d16-bde1c08b84dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:25.002293 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5ce66b09-58e9-4d98-9d16-bde1c08b84dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:25.009547 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 915/1830] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:24 2022] DELETE /identity/v3/projects/7da9159cbb5448fb9361edd9e92f2713 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:25.014452 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54600760-5d66-451d-a78a-3785564df3b4 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:25.015134 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54600760-5d66-451d-a78a-3785564df3b4 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:25.015527 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54600760-5d66-451d-a78a-3785564df3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:25.056272 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-54600760-5d66-451d-a78a-3785564df3b4 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:25.058309 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-54600760-5d66-451d-a78a-3785564df3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:25.191815 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 916/1831] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:25.200271 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479e64f8-2948-444a-9632-a1b782c76a1c None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:25.201694 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479e64f8-2948-444a-9632-a1b782c76a1c None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:25.202096 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479e64f8-2948-444a-9632-a1b782c76a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:25.244653 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:25.263900 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-479e64f8-2948-444a-9632-a1b782c76a1c None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:25.267018 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-479e64f8-2948-444a-9632-a1b782c76a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:25.297811 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 916/1832] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:25.303238 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54600760-5d66-451d-a78a-3785564df3b4 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:25.316227 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54600760-5d66-451d-a78a-3785564df3b4 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:25.338782 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40007ecc-f7d7-4d42-a30d-fbebfafb94ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:25.340201 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40007ecc-f7d7-4d42-a30d-fbebfafb94ff admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:25.341627 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40007ecc-f7d7-4d42-a30d-fbebfafb94ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:25.343251 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40007ecc-f7d7-4d42-a30d-fbebfafb94ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:25.345119 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40007ecc-f7d7-4d42-a30d-fbebfafb94ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:25.348754 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40007ecc-f7d7-4d42-a30d-fbebfafb94ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:25.352920 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40007ecc-f7d7-4d42-a30d-fbebfafb94ff admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:25.389715 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479e64f8-2948-444a-9632-a1b782c76a1c None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:25.396203 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 917/1833] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:29:25.403916 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479e64f8-2948-444a-9632-a1b782c76a1c None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:25.415464 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4186c26e-6fa5-42f5-9aae-bf8360bfa877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:25.416136 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4186c26e-6fa5-42f5-9aae-bf8360bfa877 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:25.416415 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4186c26e-6fa5-42f5-9aae-bf8360bfa877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:25.416686 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4186c26e-6fa5-42f5-9aae-bf8360bfa877 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:25.417342 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4186c26e-6fa5-42f5-9aae-bf8360bfa877 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:25.419402 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4186c26e-6fa5-42f5-9aae-bf8360bfa877 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:25.472034 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 917/1834] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:25 2022] POST /identity/v3/projects => generated 357 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:25.473775 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40007ecc-f7d7-4d42-a30d-fbebfafb94ff admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:25.493150 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40007ecc-f7d7-4d42-a30d-fbebfafb94ff admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:25.633996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11dbbb6-df72-421c-b3bf-f09bf05f46c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:25.635198 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11dbbb6-df72-421c-b3bf-f09bf05f46c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:25.635727 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11dbbb6-df72-421c-b3bf-f09bf05f46c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:25.636219 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11dbbb6-df72-421c-b3bf-f09bf05f46c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:25.637298 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11dbbb6-df72-421c-b3bf-f09bf05f46c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:25.641534 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11dbbb6-df72-421c-b3bf-f09bf05f46c7 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:25.690231 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:25.698415 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 918/1835] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:25 2022] POST /identity/v3/users => generated 279 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:25.702455 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4186c26e-6fa5-42f5-9aae-bf8360bfa877 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:25.718427 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4186c26e-6fa5-42f5-9aae-bf8360bfa877 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:25.738694 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c1949-178e-4d3a-a719-dee7fabb18e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:25.738694 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c1949-178e-4d3a-a719-dee7fabb18e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:25.739042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c1949-178e-4d3a-a719-dee7fabb18e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:25.740546 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c1949-178e-4d3a-a719-dee7fabb18e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:25.740546 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067c1949-178e-4d3a-a719-dee7fabb18e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:25.741524 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067c1949-178e-4d3a-a719-dee7fabb18e8 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:25.782870 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 918/1836] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:25.787326 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11dbbb6-df72-421c-b3bf-f09bf05f46c7 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:25.804068 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11dbbb6-df72-421c-b3bf-f09bf05f46c7 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:25.817048 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2d783-e6ae-4f9d-86e0-53f44ddec53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:25.817822 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2d783-e6ae-4f9d-86e0-53f44ddec53d admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:25.818157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2d783-e6ae-4f9d-86e0-53f44ddec53d admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:25.818844 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2d783-e6ae-4f9d-86e0-53f44ddec53d admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:25.819749 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2d783-e6ae-4f9d-86e0-53f44ddec53d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:25.822607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2d783-e6ae-4f9d-86e0-53f44ddec53d admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:25.840395 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 919/1837] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:25.844708 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c1949-178e-4d3a-a719-dee7fabb18e8 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:25.861070 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c1949-178e-4d3a-a719-dee7fabb18e8 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:25.869266 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d45b7-7fee-4d7e-9e24-bdb1679cc70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:25.870308 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d45b7-7fee-4d7e-9e24-bdb1679cc70c admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:25.870859 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d45b7-7fee-4d7e-9e24-bdb1679cc70c admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:25.871306 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d45b7-7fee-4d7e-9e24-bdb1679cc70c admin admin] PATH_INFO: `/v3/projects/db91e234121d481aa3a5a96233ba693d/users/e2492c1343d24213be564f407474d552/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:25.896717 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d45b7-7fee-4d7e-9e24-bdb1679cc70c admin admin] RBAC: Authorizing `identity:create_grant(project_id=db91e234121d481aa3a5a96233ba693d, user_id=e2492c1343d24213be564f407474d552, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:25.901195 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d45b7-7fee-4d7e-9e24-bdb1679cc70c admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:25.923001 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 919/1838] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:25 2022] PUT /identity/v3/projects/db91e234121d481aa3a5a96233ba693d/users/e2492c1343d24213be564f407474d552/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:25.939026 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c9fc0-6e29-4912-8660-8864c7b1f694 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:25.944134 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c9fc0-6e29-4912-8660-8864c7b1f694 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:25.944566 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c9fc0-6e29-4912-8660-8864c7b1f694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:25.997130 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-228c9fc0-6e29-4912-8660-8864c7b1f694 None None] MFA Rules not processed for user `e2492c1343d24213be564f407474d552`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:25.998933 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-228c9fc0-6e29-4912-8660-8864c7b1f694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:26.197648 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 920/1839] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:25 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:26.198535 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d45b7-7fee-4d7e-9e24-bdb1679cc70c admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:26.351758 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d45b7-7fee-4d7e-9e24-bdb1679cc70c admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:26.360938 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7246774-8abf-45ae-8e85-8fc8f9bc8f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:26.361579 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7246774-8abf-45ae-8e85-8fc8f9bc8f28 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:26.361859 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7246774-8abf-45ae-8e85-8fc8f9bc8f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:26.362129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7246774-8abf-45ae-8e85-8fc8f9bc8f28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:26.362958 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7246774-8abf-45ae-8e85-8fc8f9bc8f28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:26.366725 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7246774-8abf-45ae-8e85-8fc8f9bc8f28 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:26.439731 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 920/1840] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:26 2022] POST /identity/v3/projects => generated 355 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:26.443588 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c9fc0-6e29-4912-8660-8864c7b1f694 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:26.461950 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c9fc0-6e29-4912-8660-8864c7b1f694 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:26.546058 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693e965-dbbf-40a0-be49-0d5b6dfb252b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:26.547011 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693e965-dbbf-40a0-be49-0d5b6dfb252b admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:26.547297 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693e965-dbbf-40a0-be49-0d5b6dfb252b admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:26.547640 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693e965-dbbf-40a0-be49-0d5b6dfb252b admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:26.548205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f693e965-dbbf-40a0-be49-0d5b6dfb252b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:26.550278 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f693e965-dbbf-40a0-be49-0d5b6dfb252b admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:26.587532 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7246774-8abf-45ae-8e85-8fc8f9bc8f28 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:26.589330 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 921/1841] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:26 2022] POST /identity/v3/users => generated 284 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:26.608929 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7246774-8abf-45ae-8e85-8fc8f9bc8f28 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:26.625514 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a9b0e-4ee2-4f03-8f3a-69f25d23f0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:26.626835 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a9b0e-4ee2-4f03-8f3a-69f25d23f0bf admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:26.627109 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a9b0e-4ee2-4f03-8f3a-69f25d23f0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:26.627375 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a9b0e-4ee2-4f03-8f3a-69f25d23f0bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:26.628747 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a9b0e-4ee2-4f03-8f3a-69f25d23f0bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:26.630098 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a9b0e-4ee2-4f03-8f3a-69f25d23f0bf admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:26.643136 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 921/1842] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:26 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:26.646316 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693e965-dbbf-40a0-be49-0d5b6dfb252b admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:26.660622 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693e965-dbbf-40a0-be49-0d5b6dfb252b admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:26.667215 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07247e-f136-47b0-85c8-e12475823024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:26.667896 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07247e-f136-47b0-85c8-e12475823024 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:26.668173 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07247e-f136-47b0-85c8-e12475823024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:26.668441 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07247e-f136-47b0-85c8-e12475823024 admin admin] PATH_INFO: `/v3/projects/ca2a5468b46a4b788de5a483ca66a3c3/users/b7a18dff543d4a4eba4ce673b2e533d3/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:26.685006 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07247e-f136-47b0-85c8-e12475823024 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca2a5468b46a4b788de5a483ca66a3c3, user_id=b7a18dff543d4a4eba4ce673b2e533d3, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:26.687877 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07247e-f136-47b0-85c8-e12475823024 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:26.713960 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 922/1843] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:26 2022] PUT /identity/v3/projects/ca2a5468b46a4b788de5a483ca66a3c3/users/b7a18dff543d4a4eba4ce673b2e533d3/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:26.715300 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a9b0e-4ee2-4f03-8f3a-69f25d23f0bf admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:26.730424 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a9b0e-4ee2-4f03-8f3a-69f25d23f0bf admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:26.805576 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0bf10-f1fe-4da1-89c5-b6982e821d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:26.806233 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0bf10-f1fe-4da1-89c5-b6982e821d71 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:26.806510 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0bf10-f1fe-4da1-89c5-b6982e821d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:26.807081 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0bf10-f1fe-4da1-89c5-b6982e821d71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:26.807860 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef0bf10-f1fe-4da1-89c5-b6982e821d71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:26.809624 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef0bf10-f1fe-4da1-89c5-b6982e821d71 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:26.827870 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 922/1844] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:26 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:26.828598 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07247e-f136-47b0-85c8-e12475823024 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:26.841173 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07247e-f136-47b0-85c8-e12475823024 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:26.847444 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9c481c-5f00-497f-9c35-24be024f748c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:26.848572 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c481c-5f00-497f-9c35-24be024f748c admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:26.849081 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c481c-5f00-497f-9c35-24be024f748c admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:26.849527 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c481c-5f00-497f-9c35-24be024f748c admin admin] PATH_INFO: `/v3/projects/ca2a5468b46a4b788de5a483ca66a3c3/users/b7a18dff543d4a4eba4ce673b2e533d3/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:26.852233 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9c481c-5f00-497f-9c35-24be024f748c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca2a5468b46a4b788de5a483ca66a3c3, user_id=b7a18dff543d4a4eba4ce673b2e533d3, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:26.855692 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9c481c-5f00-497f-9c35-24be024f748c admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:26.874713 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 923/1845] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:26 2022] PUT /identity/v3/projects/ca2a5468b46a4b788de5a483ca66a3c3/users/b7a18dff543d4a4eba4ce673b2e533d3/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:26.901739 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c25b4-d743-4727-ae50-c3f3b2453647 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:26.902104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c25b4-d743-4727-ae50-c3f3b2453647 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:26.902385 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c25b4-d743-4727-ae50-c3f3b2453647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:26.954075 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:26.958602 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-df3c25b4-d743-4727-ae50-c3f3b2453647 None None] MFA Rules not processed for user `b7a18dff543d4a4eba4ce673b2e533d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:26.964022 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-df3c25b4-d743-4727-ae50-c3f3b2453647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:27.109485 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 923/1846] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:26 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:27.122832 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9c481c-5f00-497f-9c35-24be024f748c admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:27.124592 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-7f9c481c-5f00-497f-9c35-24be024f748c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:27.176455 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:27.216189 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9c481c-5f00-497f-9c35-24be024f748c admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:27.221118 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb46ee5-84b2-4e1b-8dfd-4ae9707bafea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:27.222448 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb46ee5-84b2-4e1b-8dfd-4ae9707bafea service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:27.222448 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb46ee5-84b2-4e1b-8dfd-4ae9707bafea service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:27.222609 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb46ee5-84b2-4e1b-8dfd-4ae9707bafea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:27.245709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb46ee5-84b2-4e1b-8dfd-4ae9707bafea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:27.247214 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb46ee5-84b2-4e1b-8dfd-4ae9707bafea service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:27.261588 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 924/1847] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:27 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:27.395854 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c25b4-d743-4727-ae50-c3f3b2453647 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:27.397196 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-df3c25b4-d743-4727-ae50-c3f3b2453647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:27.411003 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c25b4-d743-4727-ae50-c3f3b2453647 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:27.415528 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d14ac40-5d2e-41f3-8397-85fe8f7af9e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:27.416179 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d14ac40-5d2e-41f3-8397-85fe8f7af9e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:27.416452 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d14ac40-5d2e-41f3-8397-85fe8f7af9e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:27.416719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d14ac40-5d2e-41f3-8397-85fe8f7af9e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:27.494890 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d14ac40-5d2e-41f3-8397-85fe8f7af9e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:27.501901 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d14ac40-5d2e-41f3-8397-85fe8f7af9e9 service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:27.525181 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 924/1848] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:27 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:28.318214 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1aa08-5b51-47df-af92-f5514f05ce79 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:28.318624 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1aa08-5b51-47df-af92-f5514f05ce79 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:28.318728 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1aa08-5b51-47df-af92-f5514f05ce79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:28.381295 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-f4b1aa08-5b51-47df-af92-f5514f05ce79 None None] MFA Rules not processed for user `25afd7d8fcfe4dcf9133872d69520508`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:28.382761 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-f4b1aa08-5b51-47df-af92-f5514f05ce79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:28.495416 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 925/1849] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:28.501676 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d14ac40-5d2e-41f3-8397-85fe8f7af9e9 service cinder] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:28.503196 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-2d14ac40-5d2e-41f3-8397-85fe8f7af9e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:28.571307 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d14ac40-5d2e-41f3-8397-85fe8f7af9e9 service cinder] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:28.576714 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3050ae-807c-434c-bd5c-a794375c267e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:28.577340 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3050ae-807c-434c-bd5c-a794375c267e admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:28.577615 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3050ae-807c-434c-bd5c-a794375c267e admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:28.577912 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3050ae-807c-434c-bd5c-a794375c267e admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:28.578607 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3050ae-807c-434c-bd5c-a794375c267e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:28.581273 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3050ae-807c-434c-bd5c-a794375c267e admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:28.618441 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 925/1850] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:28 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:28.623615 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1aa08-5b51-47df-af92-f5514f05ce79 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:28.637759 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1aa08-5b51-47df-af92-f5514f05ce79 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:28.753430 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c7d26-51f4-4a11-b098-57a7a6f09b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:28.754327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c7d26-51f4-4a11-b098-57a7a6f09b9a admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:28.754609 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c7d26-51f4-4a11-b098-57a7a6f09b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:28.754960 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c7d26-51f4-4a11-b098-57a7a6f09b9a admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:28.755883 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5c7d26-51f4-4a11-b098-57a7a6f09b9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:28.757836 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5c7d26-51f4-4a11-b098-57a7a6f09b9a admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:28.805184 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 926/1851] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:28 2022] POST /identity/v3/users => generated 299 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:28.808832 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3050ae-807c-434c-bd5c-a794375c267e admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:28.832684 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3050ae-807c-434c-bd5c-a794375c267e admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:28.846148 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a313a347-5608-43db-ba61-3b90bbd1c904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:28.850535 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313a347-5608-43db-ba61-3b90bbd1c904 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:28.851209 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313a347-5608-43db-ba61-3b90bbd1c904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:28.851594 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313a347-5608-43db-ba61-3b90bbd1c904 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:28.852494 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a313a347-5608-43db-ba61-3b90bbd1c904 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:28.854512 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a313a347-5608-43db-ba61-3b90bbd1c904 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:28.893183 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 926/1852] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:28.894402 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c7d26-51f4-4a11-b098-57a7a6f09b9a admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:28.908225 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c7d26-51f4-4a11-b098-57a7a6f09b9a admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:28.954813 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e918e6a2-db7f-43e2-b319-0855e4b016cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:28.954813 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e918e6a2-db7f-43e2-b319-0855e4b016cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:28.954813 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e918e6a2-db7f-43e2-b319-0855e4b016cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:28.957290 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e918e6a2-db7f-43e2-b319-0855e4b016cc admin admin] PATH_INFO: `/v3/projects/890bb2756d444dbf982b606eea0e37c5/users/9adce6fc87a840e2baa33bb60d3ddcfd/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:28.998791 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e918e6a2-db7f-43e2-b319-0855e4b016cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=890bb2756d444dbf982b606eea0e37c5, user_id=9adce6fc87a840e2baa33bb60d3ddcfd, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:29.002203 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e918e6a2-db7f-43e2-b319-0855e4b016cc admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:29.040260 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 927/1853] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:28 2022] PUT /identity/v3/projects/890bb2756d444dbf982b606eea0e37c5/users/9adce6fc87a840e2baa33bb60d3ddcfd/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:29.043815 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a313a347-5608-43db-ba61-3b90bbd1c904 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:29.078947 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a313a347-5608-43db-ba61-3b90bbd1c904 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:29.106975 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:29.256814 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1851ad0d-bd5b-46c2-9da5-1660ca523813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:29.257735 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851ad0d-bd5b-46c2-9da5-1660ca523813 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:29.258016 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851ad0d-bd5b-46c2-9da5-1660ca523813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:29.258301 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851ad0d-bd5b-46c2-9da5-1660ca523813 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:29.261453 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1851ad0d-bd5b-46c2-9da5-1660ca523813 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:29.266194 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1851ad0d-bd5b-46c2-9da5-1660ca523813 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:29.292757 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 927/1854] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:29.294000 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e918e6a2-db7f-43e2-b319-0855e4b016cc admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:29.309096 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e918e6a2-db7f-43e2-b319-0855e4b016cc admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:29.314530 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584cec99-fbc5-4540-aaaa-9e144b59bbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:29.315445 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cec99-fbc5-4540-aaaa-9e144b59bbdf admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:29.315873 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cec99-fbc5-4540-aaaa-9e144b59bbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:29.316199 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cec99-fbc5-4540-aaaa-9e144b59bbdf admin admin] PATH_INFO: `/v3/projects/890bb2756d444dbf982b606eea0e37c5/users/9adce6fc87a840e2baa33bb60d3ddcfd/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:29.318560 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584cec99-fbc5-4540-aaaa-9e144b59bbdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=890bb2756d444dbf982b606eea0e37c5, user_id=9adce6fc87a840e2baa33bb60d3ddcfd, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:29.321984 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584cec99-fbc5-4540-aaaa-9e144b59bbdf admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:29.358539 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 928/1855] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:29 2022] PUT /identity/v3/projects/890bb2756d444dbf982b606eea0e37c5/users/9adce6fc87a840e2baa33bb60d3ddcfd/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:29:35.738000 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f84004e-2ff5-45a4-b796-bddde3ea690e None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:35.738510 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f84004e-2ff5-45a4-b796-bddde3ea690e None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:35.738858 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f84004e-2ff5-45a4-b796-bddde3ea690e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:35.780871 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-8f84004e-2ff5-45a4-b796-bddde3ea690e None None] MFA Rules not processed for user `9adce6fc87a840e2baa33bb60d3ddcfd`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:35.782187 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-8f84004e-2ff5-45a4-b796-bddde3ea690e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:35.958711 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 928/1856] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:35 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:38.597482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584cec99-fbc5-4540-aaaa-9e144b59bbdf admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:38.602242 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-584cec99-fbc5-4540-aaaa-9e144b59bbdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:38.635177 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:38.704229 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584cec99-fbc5-4540-aaaa-9e144b59bbdf admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:38.708929 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4245ded3-d0ac-441e-9b79-7d6e5e5a3687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:38.709565 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245ded3-d0ac-441e-9b79-7d6e5e5a3687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:38.709836 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245ded3-d0ac-441e-9b79-7d6e5e5a3687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:38.710103 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245ded3-d0ac-441e-9b79-7d6e5e5a3687 admin admin] PATH_INFO: `/v3/users/d568635a644d444096997355ceb21202` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:38.711802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4245ded3-d0ac-441e-9b79-7d6e5e5a3687 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d568635a644d444096997355ceb21202)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:38.713720 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4245ded3-d0ac-441e-9b79-7d6e5e5a3687 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:38.781342 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-4245ded3-d0ac-441e-9b79-7d6e5e5a3687 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd568635a644d444096997355ceb21202'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:38.782117 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-4245ded3-d0ac-441e-9b79-7d6e5e5a3687 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd568635a644d444096997355ceb21202'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:38.790021 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-4245ded3-d0ac-441e-9b79-7d6e5e5a3687 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd568635a644d444096997355ceb21202'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:38.803758 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-4245ded3-d0ac-441e-9b79-7d6e5e5a3687 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd568635a644d444096997355ceb21202'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:38.813957 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-4245ded3-d0ac-441e-9b79-7d6e5e5a3687 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd568635a644d444096997355ceb21202'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:38.832637 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 929/1857] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:38 2022] DELETE /identity/v3/users/d568635a644d444096997355ceb21202 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:39.116098 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f84004e-2ff5-45a4-b796-bddde3ea690e None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:39.129752 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f84004e-2ff5-45a4-b796-bddde3ea690e None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:39.220478 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4245ded3-d0ac-441e-9b79-7d6e5e5a3687 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:39.220478 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-4245ded3-d0ac-441e-9b79-7d6e5e5a3687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:39.245950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbfcf59-e9fb-4996-ab0d-64689f95ecf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:39.247114 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbfcf59-e9fb-4996-ab0d-64689f95ecf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:39.247398 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbfcf59-e9fb-4996-ab0d-64689f95ecf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:39.247673 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbfcf59-e9fb-4996-ab0d-64689f95ecf0 admin admin] PATH_INFO: `/v3/projects/9de64ee9457449bea0deedc51fd538f7` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:39.249730 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbfcf59-e9fb-4996-ab0d-64689f95ecf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9de64ee9457449bea0deedc51fd538f7)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:39.258831 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbfcf59-e9fb-4996-ab0d-64689f95ecf0 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:39.327594 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:39.351121 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-afbfcf59-e9fb-4996-ab0d-64689f95ecf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9de64ee9457449bea0deedc51fd538f7', 'request_id': 'req-afbfcf59-e9fb-4996-ab0d-64689f95ecf0'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:39.361305 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4245ded3-d0ac-441e-9b79-7d6e5e5a3687 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:39.371462 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-afbfcf59-e9fb-4996-ab0d-64689f95ecf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9de64ee9457449bea0deedc51fd538f7', 'request_id': 'req-afbfcf59-e9fb-4996-ab0d-64689f95ecf0'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:39.401144 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-afbfcf59-e9fb-4996-ab0d-64689f95ecf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9de64ee9457449bea0deedc51fd538f7', 'request_id': 'req-afbfcf59-e9fb-4996-ab0d-64689f95ecf0'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:39.444891 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-afbfcf59-e9fb-4996-ab0d-64689f95ecf0 admin admin] The token cache is being invalidate because project 9de64ee9457449bea0deedc51fd538f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:39.445202 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-afbfcf59-e9fb-4996-ab0d-64689f95ecf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:39.447619 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-afbfcf59-e9fb-4996-ab0d-64689f95ecf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:39.457365 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 929/1858] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:39 2022] DELETE /identity/v3/projects/9de64ee9457449bea0deedc51fd538f7 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:39.458833 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159188c-e60a-41d4-8377-3ed2f07413c4 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:39.458833 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159188c-e60a-41d4-8377-3ed2f07413c4 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:39.462590 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159188c-e60a-41d4-8377-3ed2f07413c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:39.492979 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a278e62-0b9e-43a0-a0a7-9c67ade6fecf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a051ca8154540ad93db6d028b8323a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:39.493944 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a278e62-0b9e-43a0-a0a7-9c67ade6fecf service glance] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:39.494456 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a278e62-0b9e-43a0-a0a7-9c67ade6fecf service glance] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:39.494908 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a278e62-0b9e-43a0-a0a7-9c67ade6fecf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:39.497985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-4a278e62-0b9e-43a0-a0a7-9c67ade6fecf service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:39.510374 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:39.522368 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-f159188c-e60a-41d4-8377-3ed2f07413c4 None None] MFA Rules not processed for user `364555ec385c4dbb91907fb9c5116d2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:39.527272 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-f159188c-e60a-41d4-8377-3ed2f07413c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:39.630326 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a278e62-0b9e-43a0-a0a7-9c67ade6fecf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:39.631924 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a278e62-0b9e-43a0-a0a7-9c67ade6fecf service glance] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:39.638702 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 930/1859] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Nov 2 07:29:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:39.647730 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42a251-08b8-4432-8202-804f2d092bd6 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:39.648954 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42a251-08b8-4432-8202-804f2d092bd6 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:39.648954 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42a251-08b8-4432-8202-804f2d092bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:39.659428 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 930/1860] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:29:39.684139 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-ce42a251-08b8-4432-8202-804f2d092bd6 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:39.685373 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-ce42a251-08b8-4432-8202-804f2d092bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:39.702137 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a278e62-0b9e-43a0-a0a7-9c67ade6fecf service glance] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:39.703912 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-4a278e62-0b9e-43a0-a0a7-9c67ade6fecf service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:39.787604 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 931/1861] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:39.795259 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484a938-22cb-4b4f-9ae7-9cb1c6c14284 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:39.795541 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484a938-22cb-4b4f-9ae7-9cb1c6c14284 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:39.795828 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484a938-22cb-4b4f-9ae7-9cb1c6c14284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:39.801138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a278e62-0b9e-43a0-a0a7-9c67ade6fecf service glance] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:39.817441 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4bf0fb-d9cf-498e-aa75-fbfacb72b6ca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '836c4044aaf74de391219289b57d2956', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:39.818283 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4bf0fb-d9cf-498e-aa75-fbfacb72b6ca service swift] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:39.818852 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4bf0fb-d9cf-498e-aa75-fbfacb72b6ca service swift] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:39.819260 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4bf0fb-d9cf-498e-aa75-fbfacb72b6ca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:39.847019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-8484a938-22cb-4b4f-9ae7-9cb1c6c14284 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:39.850120 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-8484a938-22cb-4b4f-9ae7-9cb1c6c14284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:39.852731 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4bf0fb-d9cf-498e-aa75-fbfacb72b6ca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:39.854295 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4bf0fb-d9cf-498e-aa75-fbfacb72b6ca service swift] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:39.869996 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 932/1862] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:39.874163 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8484a938-22cb-4b4f-9ae7-9cb1c6c14284 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:39.887218 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 931/1863] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Nov 2 07:29:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Nov 02 07:29:39.897237 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8484a938-22cb-4b4f-9ae7-9cb1c6c14284 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:39.913473 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a89b9-4298-47eb-87a6-a5c3393db714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:39.913473 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a89b9-4298-47eb-87a6-a5c3393db714 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:39.913691 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a89b9-4298-47eb-87a6-a5c3393db714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:39.913850 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a89b9-4298-47eb-87a6-a5c3393db714 admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:39.914703 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a89b9-4298-47eb-87a6-a5c3393db714 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:39.914999 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a89b9-4298-47eb-87a6-a5c3393db714 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:39.929317 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a89b9-4298-47eb-87a6-a5c3393db714 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:39.931846 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79bd7e6-18ca-4566-b7b8-2626117ee93b None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:39.935107 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79bd7e6-18ca-4566-b7b8-2626117ee93b None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:39.935496 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79bd7e6-18ca-4566-b7b8-2626117ee93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:39.958705 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 933/1864] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:29:39.959927 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a89b9-4298-47eb-87a6-a5c3393db714 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:39.982882 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-b79bd7e6-18ca-4566-b7b8-2626117ee93b None None] MFA Rules not processed for user `364555ec385c4dbb91907fb9c5116d2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:39.987648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a89b9-4298-47eb-87a6-a5c3393db714 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:39.988438 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-b79bd7e6-18ca-4566-b7b8-2626117ee93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:39.994561 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a02852-436f-4522-8fcb-a2519ab6f244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:39.996240 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a02852-436f-4522-8fcb-a2519ab6f244 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:39.996704 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a02852-436f-4522-8fcb-a2519ab6f244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:39.997208 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a02852-436f-4522-8fcb-a2519ab6f244 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:39.998466 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a02852-436f-4522-8fcb-a2519ab6f244 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:40.003545 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a02852-436f-4522-8fcb-a2519ab6f244 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:40.007979 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 932/1865] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Nov 2 07:29:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:40.020839 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79bd7e6-18ca-4566-b7b8-2626117ee93b None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:40.041502 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79bd7e6-18ca-4566-b7b8-2626117ee93b None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:40.050878 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e7dd2-514d-4ea6-b948-fe377c039ff3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '836c4044aaf74de391219289b57d2956', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:40.051606 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e7dd2-514d-4ea6-b948-fe377c039ff3 service swift] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:40.052001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e7dd2-514d-4ea6-b948-fe377c039ff3 service swift] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:40.052379 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e7dd2-514d-4ea6-b948-fe377c039ff3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:40.065997 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0e7dd2-514d-4ea6-b948-fe377c039ff3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:40.070185 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0e7dd2-514d-4ea6-b948-fe377c039ff3 service swift] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:40.075442 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 934/1866] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:39 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:29:40.081267 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a02852-436f-4522-8fcb-a2519ab6f244 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:40.096997 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 933/1867] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Nov 2 07:29:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Nov 02 07:29:40.116981 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:40.162131 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a02852-436f-4522-8fcb-a2519ab6f244 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:40.169959 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3d575-c381-417b-836e-235f9ffceef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:40.172112 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3d575-c381-417b-836e-235f9ffceef2 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:40.174725 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3d575-c381-417b-836e-235f9ffceef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:40.175479 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3d575-c381-417b-836e-235f9ffceef2 admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:40.176318 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3d575-c381-417b-836e-235f9ffceef2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:40.180043 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3d575-c381-417b-836e-235f9ffceef2 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:40.216183 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 935/1868] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:40 2022] POST /identity/v3/users => generated 293 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:40.220892 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e7dd2-514d-4ea6-b948-fe377c039ff3 service swift] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:40.268683 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e7dd2-514d-4ea6-b948-fe377c039ff3 service swift] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:40.284220 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79670db8-2b8b-4f05-8c36-c030ce956db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:40.285190 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79670db8-2b8b-4f05-8c36-c030ce956db2 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:40.285828 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79670db8-2b8b-4f05-8c36-c030ce956db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:40.286216 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79670db8-2b8b-4f05-8c36-c030ce956db2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:40.287321 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79670db8-2b8b-4f05-8c36-c030ce956db2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:40.289678 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79670db8-2b8b-4f05-8c36-c030ce956db2 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:40.312783 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 934/1869] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:40.318441 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3d575-c381-417b-836e-235f9ffceef2 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:40.345484 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3d575-c381-417b-836e-235f9ffceef2 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:40.351143 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2c5c2-2a2e-4440-a14a-684d4fa6ad62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:40.351780 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2c5c2-2a2e-4440-a14a-684d4fa6ad62 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:40.352059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2c5c2-2a2e-4440-a14a-684d4fa6ad62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:40.352326 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2c5c2-2a2e-4440-a14a-684d4fa6ad62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:40.353228 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f2c5c2-2a2e-4440-a14a-684d4fa6ad62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:40.355115 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f2c5c2-2a2e-4440-a14a-684d4fa6ad62 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:40.370762 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 936/1870] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:40.372509 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79670db8-2b8b-4f05-8c36-c030ce956db2 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:40.402979 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79670db8-2b8b-4f05-8c36-c030ce956db2 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:40.415295 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36523ef-8df5-47d0-84c8-c6ac2e1eccd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:40.415983 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36523ef-8df5-47d0-84c8-c6ac2e1eccd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:40.416260 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36523ef-8df5-47d0-84c8-c6ac2e1eccd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:40.416772 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36523ef-8df5-47d0-84c8-c6ac2e1eccd5 admin admin] PATH_INFO: `/v3/projects/dd9dd4a58fc844c5b2837b4bf3c2ef3b/users/d9f6c8cb61a244c6b6050a22b0c4ed00/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:40.445289 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36523ef-8df5-47d0-84c8-c6ac2e1eccd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd9dd4a58fc844c5b2837b4bf3c2ef3b, user_id=d9f6c8cb61a244c6b6050a22b0c4ed00, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:40.448387 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36523ef-8df5-47d0-84c8-c6ac2e1eccd5 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:40.473010 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 935/1871] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:40 2022] PUT /identity/v3/projects/dd9dd4a58fc844c5b2837b4bf3c2ef3b/users/d9f6c8cb61a244c6b6050a22b0c4ed00/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:40.484666 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9509e17-633a-487c-851a-5498cf8e9559 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:40.485140 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9509e17-633a-487c-851a-5498cf8e9559 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:40.485599 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9509e17-633a-487c-851a-5498cf8e9559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:40.581922 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-b9509e17-633a-487c-851a-5498cf8e9559 None None] MFA Rules not processed for user `d9f6c8cb61a244c6b6050a22b0c4ed00`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:40.587312 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-b9509e17-633a-487c-851a-5498cf8e9559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:40.750141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 937/1872] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:40.768004 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36523ef-8df5-47d0-84c8-c6ac2e1eccd5 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:40.770092 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-c36523ef-8df5-47d0-84c8-c6ac2e1eccd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:40.878410 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:40.962685 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36523ef-8df5-47d0-84c8-c6ac2e1eccd5 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:40.971517 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53719f96-1b84-4145-8c84-4b03cbd14cd1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:40.974999 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53719f96-1b84-4145-8c84-4b03cbd14cd1 service nova] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:40.975658 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53719f96-1b84-4145-8c84-4b03cbd14cd1 service nova] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:40.976124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53719f96-1b84-4145-8c84-4b03cbd14cd1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:40.989775 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53719f96-1b84-4145-8c84-4b03cbd14cd1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:40.991908 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53719f96-1b84-4145-8c84-4b03cbd14cd1 service nova] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:41.007863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 936/1873] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:41.136179 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32added-104c-427f-be9d-fdf11b62a8b4 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:41.137086 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32added-104c-427f-be9d-fdf11b62a8b4 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:41.137362 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32added-104c-427f-be9d-fdf11b62a8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:41.166022 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-b32added-104c-427f-be9d-fdf11b62a8b4 None None] MFA Rules not processed for user `364555ec385c4dbb91907fb9c5116d2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:41.167614 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-b32added-104c-427f-be9d-fdf11b62a8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:41.274786 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 938/1874] 10.222.0.30 () {58 vars in 967 bytes} [Wed Nov 2 07:29:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:41.334424 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53719f96-1b84-4145-8c84-4b03cbd14cd1 service nova] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:41.449547 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53719f96-1b84-4145-8c84-4b03cbd14cd1 service nova] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:41.466642 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51358e9b-4ec1-41ab-aec1-49eb5fe987b4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '836c4044aaf74de391219289b57d2956', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:41.468414 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51358e9b-4ec1-41ab-aec1-49eb5fe987b4 service swift] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:41.468850 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51358e9b-4ec1-41ab-aec1-49eb5fe987b4 service swift] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:41.469194 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51358e9b-4ec1-41ab-aec1-49eb5fe987b4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:41.495265 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51358e9b-4ec1-41ab-aec1-49eb5fe987b4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:41.497037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51358e9b-4ec1-41ab-aec1-49eb5fe987b4 service swift] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:41.515187 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 937/1875] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Nov 2 07:29:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:29:42.048757 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b1d3d-143e-42f5-9e32-eeffe295de43 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:42.049042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b1d3d-143e-42f5-9e32-eeffe295de43 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:42.049481 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b1d3d-143e-42f5-9e32-eeffe295de43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:42.079923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-e40b1d3d-143e-42f5-9e32-eeffe295de43 None None] MFA Rules not processed for user `364555ec385c4dbb91907fb9c5116d2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:42.081229 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-e40b1d3d-143e-42f5-9e32-eeffe295de43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:42.102810 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 939/1876] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Nov 2 07:29:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:42.118517 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51358e9b-4ec1-41ab-aec1-49eb5fe987b4 service swift] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:42.130743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51358e9b-4ec1-41ab-aec1-49eb5fe987b4 service swift] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:42.138586 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1520f130-8219-48cd-9f74-d4d995ed513a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '836c4044aaf74de391219289b57d2956', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:42.139391 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1520f130-8219-48cd-9f74-d4d995ed513a service swift] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:42.139680 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1520f130-8219-48cd-9f74-d4d995ed513a service swift] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:42.139953 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1520f130-8219-48cd-9f74-d4d995ed513a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:42.149296 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1520f130-8219-48cd-9f74-d4d995ed513a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:42.151121 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1520f130-8219-48cd-9f74-d4d995ed513a service swift] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:42.173075 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 938/1877] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Nov 2 07:29:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:29:42.692335 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68758276-dec9-4d1e-9077-b68367eb1e9d None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:42.692923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68758276-dec9-4d1e-9077-b68367eb1e9d None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:42.692923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68758276-dec9-4d1e-9077-b68367eb1e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:42.717268 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-68758276-dec9-4d1e-9077-b68367eb1e9d None None] MFA Rules not processed for user `364555ec385c4dbb91907fb9c5116d2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:42.718509 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-68758276-dec9-4d1e-9077-b68367eb1e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:42.729456 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 940/1878] 10.222.0.30 () {58 vars in 967 bytes} [Wed Nov 2 07:29:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:42.745302 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1520f130-8219-48cd-9f74-d4d995ed513a service swift] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:42.768412 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1520f130-8219-48cd-9f74-d4d995ed513a service swift] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:42.775674 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decbe471-2ace-4334-88e2-495e956e8cd2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '836c4044aaf74de391219289b57d2956', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:42.776436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decbe471-2ace-4334-88e2-495e956e8cd2 service swift] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:42.776776 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decbe471-2ace-4334-88e2-495e956e8cd2 service swift] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:42.777112 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decbe471-2ace-4334-88e2-495e956e8cd2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:42.786759 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decbe471-2ace-4334-88e2-495e956e8cd2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:42.788482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decbe471-2ace-4334-88e2-495e956e8cd2 service swift] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:42.799504 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 939/1879] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Nov 2 07:29:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:29:48.751785 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68758276-dec9-4d1e-9077-b68367eb1e9d None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:48.754087 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-68758276-dec9-4d1e-9077-b68367eb1e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:48.817296 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:48.887901 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68758276-dec9-4d1e-9077-b68367eb1e9d None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:48.893054 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5968df-d17f-4f65-8d81-6f25498741d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:48.894612 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5968df-d17f-4f65-8d81-6f25498741d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:48.895083 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5968df-d17f-4f65-8d81-6f25498741d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:48.895633 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5968df-d17f-4f65-8d81-6f25498741d3 admin admin] PATH_INFO: `/v3/users/e232cf99f83a4501afcbc07686f10904` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:48.897166 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5968df-d17f-4f65-8d81-6f25498741d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e232cf99f83a4501afcbc07686f10904)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:48.899474 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5968df-d17f-4f65-8d81-6f25498741d3 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:48.986568 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5d5968df-d17f-4f65-8d81-6f25498741d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e232cf99f83a4501afcbc07686f10904'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:49.002210 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5d5968df-d17f-4f65-8d81-6f25498741d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e232cf99f83a4501afcbc07686f10904'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:49.003251 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5d5968df-d17f-4f65-8d81-6f25498741d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e232cf99f83a4501afcbc07686f10904'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:49.026485 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5d5968df-d17f-4f65-8d81-6f25498741d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e232cf99f83a4501afcbc07686f10904'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:49.053490 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5d5968df-d17f-4f65-8d81-6f25498741d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e232cf99f83a4501afcbc07686f10904'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:49.077141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 941/1880] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:48 2022] DELETE /identity/v3/users/e232cf99f83a4501afcbc07686f10904 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:49.081436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2967d8e-e0fb-4b1a-9b82-c378de454712 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:49.082037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2967d8e-e0fb-4b1a-9b82-c378de454712 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:49.082133 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2967d8e-e0fb-4b1a-9b82-c378de454712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:49.120054 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-d2967d8e-e0fb-4b1a-9b82-c378de454712 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:49.121305 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-d2967d8e-e0fb-4b1a-9b82-c378de454712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:49.234690 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 940/1881] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:49.259085 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5968df-d17f-4f65-8d81-6f25498741d3 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:49.260764 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-5d5968df-d17f-4f65-8d81-6f25498741d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:49.396734 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5968df-d17f-4f65-8d81-6f25498741d3 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:49.412265 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528ba1c-3930-4c36-93cf-017a8d3ff89b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:49.412265 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528ba1c-3930-4c36-93cf-017a8d3ff89b service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:49.412265 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528ba1c-3930-4c36-93cf-017a8d3ff89b service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:49.413674 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528ba1c-3930-4c36-93cf-017a8d3ff89b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:49.427038 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528ba1c-3930-4c36-93cf-017a8d3ff89b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:49.428666 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528ba1c-3930-4c36-93cf-017a8d3ff89b service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:49.446177 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 942/1882] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:49.714495 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2967d8e-e0fb-4b1a-9b82-c378de454712 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:49.716476 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-d2967d8e-e0fb-4b1a-9b82-c378de454712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:49.737804 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2967d8e-e0fb-4b1a-9b82-c378de454712 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:49.769048 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a40fd-2dcc-4f69-b2bf-bc3e7d221860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:49.771830 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a40fd-2dcc-4f69-b2bf-bc3e7d221860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:49.772733 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a40fd-2dcc-4f69-b2bf-bc3e7d221860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:49.773859 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a40fd-2dcc-4f69-b2bf-bc3e7d221860 admin admin] PATH_INFO: `/v3/projects/6df0bcee2fc949c789b67b2e64b06cd0` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:49.775849 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a40fd-2dcc-4f69-b2bf-bc3e7d221860 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6df0bcee2fc949c789b67b2e64b06cd0)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:49.782224 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a40fd-2dcc-4f69-b2bf-bc3e7d221860 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:49.838005 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:49.885372 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-2d7a40fd-2dcc-4f69-b2bf-bc3e7d221860 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6df0bcee2fc949c789b67b2e64b06cd0', 'request_id': 'req-2d7a40fd-2dcc-4f69-b2bf-bc3e7d221860'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:49.894231 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-2d7a40fd-2dcc-4f69-b2bf-bc3e7d221860 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6df0bcee2fc949c789b67b2e64b06cd0', 'request_id': 'req-2d7a40fd-2dcc-4f69-b2bf-bc3e7d221860'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:49.905020 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-2d7a40fd-2dcc-4f69-b2bf-bc3e7d221860 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6df0bcee2fc949c789b67b2e64b06cd0', 'request_id': 'req-2d7a40fd-2dcc-4f69-b2bf-bc3e7d221860'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:49.927823 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-2d7a40fd-2dcc-4f69-b2bf-bc3e7d221860 admin admin] The token cache is being invalidate because project 6df0bcee2fc949c789b67b2e64b06cd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:49.928400 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-2d7a40fd-2dcc-4f69-b2bf-bc3e7d221860 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:49.929163 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-2d7a40fd-2dcc-4f69-b2bf-bc3e7d221860 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:49.931288 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 941/1883] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:49 2022] DELETE /identity/v3/projects/6df0bcee2fc949c789b67b2e64b06cd0 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:49.937413 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d924186a-8e71-4002-abbb-e0a0b88411e7 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:49.938013 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d924186a-8e71-4002-abbb-e0a0b88411e7 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:49.938013 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d924186a-8e71-4002-abbb-e0a0b88411e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:49.988942 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-d924186a-8e71-4002-abbb-e0a0b88411e7 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:49.994874 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-d924186a-8e71-4002-abbb-e0a0b88411e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:50.075796 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a40fd-2dcc-4f69-b2bf-bc3e7d221860 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:50.077284 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-2d7a40fd-2dcc-4f69-b2bf-bc3e7d221860 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:50.129023 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:50.155146 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 943/1884] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:50.162000 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87b631-18c9-4940-b01c-788d82432be4 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.162291 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87b631-18c9-4940-b01c-788d82432be4 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.162556 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87b631-18c9-4940-b01c-788d82432be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.183638 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a40fd-2dcc-4f69-b2bf-bc3e7d221860 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:50.184201 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-bc87b631-18c9-4940-b01c-788d82432be4 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:50.185597 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-bc87b631-18c9-4940-b01c-788d82432be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:50.191540 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98376fbf-edba-4728-b295-3434fdc1cc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:50.191540 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98376fbf-edba-4728-b295-3434fdc1cc86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.191540 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98376fbf-edba-4728-b295-3434fdc1cc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.191540 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98376fbf-edba-4728-b295-3434fdc1cc86 admin admin] PATH_INFO: `/v3/users/e2492c1343d24213be564f407474d552` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.196626 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98376fbf-edba-4728-b295-3434fdc1cc86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2492c1343d24213be564f407474d552)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:50.196626 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98376fbf-edba-4728-b295-3434fdc1cc86 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:50.199747 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 944/1885] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:50.200756 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc87b631-18c9-4940-b01c-788d82432be4 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:50.228862 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc87b631-18c9-4940-b01c-788d82432be4 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:50.245696 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36defb6-9e0f-4637-9e0a-57df8e993f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:50.246636 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36defb6-9e0f-4637-9e0a-57df8e993f42 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.247152 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36defb6-9e0f-4637-9e0a-57df8e993f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.247841 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36defb6-9e0f-4637-9e0a-57df8e993f42 admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.248686 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36defb6-9e0f-4637-9e0a-57df8e993f42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:50.249176 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36defb6-9e0f-4637-9e0a-57df8e993f42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:50.251635 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36defb6-9e0f-4637-9e0a-57df8e993f42 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:50.269735 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 945/1886] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:29:50.282504 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36defb6-9e0f-4637-9e0a-57df8e993f42 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:50.293582 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36defb6-9e0f-4637-9e0a-57df8e993f42 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:50.299241 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8e1eed-7f92-4adb-80b6-825575155d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:50.299874 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e1eed-7f92-4adb-80b6-825575155d28 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.300143 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e1eed-7f92-4adb-80b6-825575155d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.300624 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e1eed-7f92-4adb-80b6-825575155d28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.301510 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8e1eed-7f92-4adb-80b6-825575155d28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:50.302894 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-98376fbf-edba-4728-b295-3434fdc1cc86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2492c1343d24213be564f407474d552'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:50.303894 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8e1eed-7f92-4adb-80b6-825575155d28 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:50.304027 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-98376fbf-edba-4728-b295-3434fdc1cc86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2492c1343d24213be564f407474d552'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:50.320166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-98376fbf-edba-4728-b295-3434fdc1cc86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2492c1343d24213be564f407474d552'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:50.345571 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-98376fbf-edba-4728-b295-3434fdc1cc86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2492c1343d24213be564f407474d552'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:50.353267 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 946/1887] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:50 2022] POST /identity/v3/projects => generated 377 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:50.354375 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8e1eed-7f92-4adb-80b6-825575155d28 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:50.356166 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-8d8e1eed-7f92-4adb-80b6-825575155d28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:50.360377 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-98376fbf-edba-4728-b295-3434fdc1cc86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2492c1343d24213be564f407474d552'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:50.371523 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 942/1888] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:50 2022] DELETE /identity/v3/users/e2492c1343d24213be564f407474d552 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:29:50.372748 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98376fbf-edba-4728-b295-3434fdc1cc86 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:50.374547 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-98376fbf-edba-4728-b295-3434fdc1cc86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:50.423117 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:50.469659 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98376fbf-edba-4728-b295-3434fdc1cc86 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:50.475080 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cfe512-85cb-4ae0-b467-3d5e04ac0308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:50.475793 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfe512-85cb-4ae0-b467-3d5e04ac0308 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.476175 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfe512-85cb-4ae0-b467-3d5e04ac0308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.476541 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfe512-85cb-4ae0-b467-3d5e04ac0308 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.477273 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cfe512-85cb-4ae0-b467-3d5e04ac0308 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:50.479593 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cfe512-85cb-4ae0-b467-3d5e04ac0308 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:50.502257 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8e1eed-7f92-4adb-80b6-825575155d28 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:50.508726 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6639d11-abab-4b53-9468-00c39ef65d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:50.509875 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6639d11-abab-4b53-9468-00c39ef65d58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.510150 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6639d11-abab-4b53-9468-00c39ef65d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.510645 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6639d11-abab-4b53-9468-00c39ef65d58 admin admin] PATH_INFO: `/v3/users/d9f6c8cb61a244c6b6050a22b0c4ed00` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.512537 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6639d11-abab-4b53-9468-00c39ef65d58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9f6c8cb61a244c6b6050a22b0c4ed00)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:50.517658 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 943/1889] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:50 2022] POST /identity/v3/users => generated 289 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:50.517658 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f158c-b775-4397-adbe-d733e739e86b None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.517658 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f158c-b775-4397-adbe-d733e739e86b None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.517658 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f158c-b775-4397-adbe-d733e739e86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.519219 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6639d11-abab-4b53-9468-00c39ef65d58 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:50.565002 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-994f158c-b775-4397-adbe-d733e739e86b None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:50.567082 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-994f158c-b775-4397-adbe-d733e739e86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:50.583372 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 944/1890] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:50.584741 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f158c-b775-4397-adbe-d733e739e86b None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:50.622000 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f158c-b775-4397-adbe-d733e739e86b None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:50.624733 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-e6639d11-abab-4b53-9468-00c39ef65d58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9f6c8cb61a244c6b6050a22b0c4ed00'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:50.635948 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac1cd66-ebfb-4bfd-85c0-864771665573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:50.636363 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac1cd66-ebfb-4bfd-85c0-864771665573 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.636945 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac1cd66-ebfb-4bfd-85c0-864771665573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.637075 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac1cd66-ebfb-4bfd-85c0-864771665573 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.637741 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac1cd66-ebfb-4bfd-85c0-864771665573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:50.639482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac1cd66-ebfb-4bfd-85c0-864771665573 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:50.642039 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-e6639d11-abab-4b53-9468-00c39ef65d58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9f6c8cb61a244c6b6050a22b0c4ed00'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:50.642851 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-e6639d11-abab-4b53-9468-00c39ef65d58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9f6c8cb61a244c6b6050a22b0c4ed00'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:50.660721 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 945/1891] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:50.660721 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac1cd66-ebfb-4bfd-85c0-864771665573 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:50.667747 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-2ac1cd66-ebfb-4bfd-85c0-864771665573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:50.676494 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-e6639d11-abab-4b53-9468-00c39ef65d58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9f6c8cb61a244c6b6050a22b0c4ed00'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:50.687522 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-e6639d11-abab-4b53-9468-00c39ef65d58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9f6c8cb61a244c6b6050a22b0c4ed00'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:50.698874 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 947/1892] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:50 2022] DELETE /identity/v3/users/d9f6c8cb61a244c6b6050a22b0c4ed00 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:50.699520 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6639d11-abab-4b53-9468-00c39ef65d58 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:50.703403 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-e6639d11-abab-4b53-9468-00c39ef65d58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:50.738447 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:50.741384 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac1cd66-ebfb-4bfd-85c0-864771665573 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:50.781660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6639d11-abab-4b53-9468-00c39ef65d58 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:50.786686 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb024e5-787f-4d0b-925e-c98dabeb15c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:50.787644 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb024e5-787f-4d0b-925e-c98dabeb15c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.789523 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb024e5-787f-4d0b-925e-c98dabeb15c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.789815 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb024e5-787f-4d0b-925e-c98dabeb15c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.790477 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb024e5-787f-4d0b-925e-c98dabeb15c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:50.792168 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb024e5-787f-4d0b-925e-c98dabeb15c8 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:50.808613 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 948/1893] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:50.813077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c1a29-a2bf-4641-a095-7cdc62fb14f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:50.813234 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea6de7-a19d-4c70-bc8f-e60f797248d0 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.813450 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea6de7-a19d-4c70-bc8f-e60f797248d0 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.813678 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea6de7-a19d-4c70-bc8f-e60f797248d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.813779 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c1a29-a2bf-4641-a095-7cdc62fb14f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.814005 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c1a29-a2bf-4641-a095-7cdc62fb14f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.814279 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c1a29-a2bf-4641-a095-7cdc62fb14f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.816245 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-ad7c1a29-a2bf-4641-a095-7cdc62fb14f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:50.835867 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7c1a29-a2bf-4641-a095-7cdc62fb14f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:50.837216 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7c1a29-a2bf-4641-a095-7cdc62fb14f5 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:50.837343 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-42ea6de7-a19d-4c70-bc8f-e60f797248d0 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:50.838831 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-42ea6de7-a19d-4c70-bc8f-e60f797248d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:50.854705 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 946/1894] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:50.854705 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c1a29-a2bf-4641-a095-7cdc62fb14f5 service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:50.855007 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 949/1895] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:50.864519 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea6de7-a19d-4c70-bc8f-e60f797248d0 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:50.864957 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c1a29-a2bf-4641-a095-7cdc62fb14f5 service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:50.867238 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-42ea6de7-a19d-4c70-bc8f-e60f797248d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:50.870114 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ebdcc8-3e78-4001-a7c4-846b735a8ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:50.870928 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ebdcc8-3e78-4001-a7c4-846b735a8ad6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.871270 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ebdcc8-3e78-4001-a7c4-846b735a8ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.871628 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ebdcc8-3e78-4001-a7c4-846b735a8ad6 admin admin] PATH_INFO: `/v3/projects/e492702cf15848cda2fa93e1db845c25/users/a47b81144eba4283aac610c600e73ed3/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.896081 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea6de7-a19d-4c70-bc8f-e60f797248d0 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:50.896326 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ebdcc8-3e78-4001-a7c4-846b735a8ad6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e492702cf15848cda2fa93e1db845c25, user_id=a47b81144eba4283aac610c600e73ed3, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:50.898978 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ebdcc8-3e78-4001-a7c4-846b735a8ad6 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:50.900380 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740296d-57cf-43b0-9612-068964e14ba0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:50.901099 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740296d-57cf-43b0-9612-068964e14ba0 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.902125 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740296d-57cf-43b0-9612-068964e14ba0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.902840 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740296d-57cf-43b0-9612-068964e14ba0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.918095 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e740296d-57cf-43b0-9612-068964e14ba0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:50.919633 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e740296d-57cf-43b0-9612-068964e14ba0 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:50.929593 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 947/1896] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:50 2022] PUT /identity/v3/projects/e492702cf15848cda2fa93e1db845c25/users/a47b81144eba4283aac610c600e73ed3/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:50.934968 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42734bd5-6f00-41ac-8c32-2a91a60608b8 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:50.934968 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42734bd5-6f00-41ac-8c32-2a91a60608b8 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:50.935190 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42734bd5-6f00-41ac-8c32-2a91a60608b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:50.942502 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 950/1897] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:50.957975 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-42734bd5-6f00-41ac-8c32-2a91a60608b8 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:50.959744 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-42734bd5-6f00-41ac-8c32-2a91a60608b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:51.004580 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:51.046139 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 948/1898] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:51.061403 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740296d-57cf-43b0-9612-068964e14ba0 service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:51.063084 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-e740296d-57cf-43b0-9612-068964e14ba0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:51.075845 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740296d-57cf-43b0-9612-068964e14ba0 service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:51.151494 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d389d59-b136-4a83-9acd-895a561b6cd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:51.152423 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d389d59-b136-4a83-9acd-895a561b6cd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:51.152900 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d389d59-b136-4a83-9acd-895a561b6cd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:51.153374 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d389d59-b136-4a83-9acd-895a561b6cd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:51.166173 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d389d59-b136-4a83-9acd-895a561b6cd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:51.168045 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d389d59-b136-4a83-9acd-895a561b6cd0 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:51.180016 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 951/1899] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:51.186381 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca15d10-ff65-4419-93ca-1d70ae954b7a None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:51.186908 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca15d10-ff65-4419-93ca-1d70ae954b7a None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:51.187015 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca15d10-ff65-4419-93ca-1d70ae954b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:51.208505 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-fca15d10-ff65-4419-93ca-1d70ae954b7a None None] MFA Rules not processed for user `25afd7d8fcfe4dcf9133872d69520508`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:51.209824 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-fca15d10-ff65-4419-93ca-1d70ae954b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:51.278109 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 949/1900] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:51.392172 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d389d59-b136-4a83-9acd-895a561b6cd0 service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:51.395205 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-3d389d59-b136-4a83-9acd-895a561b6cd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:51.416155 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d389d59-b136-4a83-9acd-895a561b6cd0 service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:51.423949 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d65e1-feea-4298-a588-eb63cadb39ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:51.425371 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d65e1-feea-4298-a588-eb63cadb39ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:51.425842 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d65e1-feea-4298-a588-eb63cadb39ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:51.426297 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d65e1-feea-4298-a588-eb63cadb39ec admin admin] PATH_INFO: `/v3/projects/db91e234121d481aa3a5a96233ba693d` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:51.442399 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d65e1-feea-4298-a588-eb63cadb39ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=db91e234121d481aa3a5a96233ba693d)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:51.447476 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d65e1-feea-4298-a588-eb63cadb39ec admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:51.459013 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca15d10-ff65-4419-93ca-1d70ae954b7a None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:51.460538 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-fca15d10-ff65-4419-93ca-1d70ae954b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:51.507109 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca15d10-ff65-4419-93ca-1d70ae954b7a None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:51.528611 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c3960-d943-4d1e-a81a-2efcad8660ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:51.528611 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c3960-d943-4d1e-a81a-2efcad8660ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:51.530388 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c3960-d943-4d1e-a81a-2efcad8660ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:51.532811 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c3960-d943-4d1e-a81a-2efcad8660ce admin admin] PATH_INFO: `/v3/projects/dd9dd4a58fc844c5b2837b4bf3c2ef3b` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:51.535323 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848c3960-d943-4d1e-a81a-2efcad8660ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd9dd4a58fc844c5b2837b4bf3c2ef3b)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:51.537924 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848c3960-d943-4d1e-a81a-2efcad8660ce admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:51.628986 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-9e5d65e1-feea-4298-a588-eb63cadb39ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db91e234121d481aa3a5a96233ba693d', 'request_id': 'req-9e5d65e1-feea-4298-a588-eb63cadb39ec'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:51.642460 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-9e5d65e1-feea-4298-a588-eb63cadb39ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db91e234121d481aa3a5a96233ba693d', 'request_id': 'req-9e5d65e1-feea-4298-a588-eb63cadb39ec'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:51.654005 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-9e5d65e1-feea-4298-a588-eb63cadb39ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db91e234121d481aa3a5a96233ba693d', 'request_id': 'req-9e5d65e1-feea-4298-a588-eb63cadb39ec'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:51.674465 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-9e5d65e1-feea-4298-a588-eb63cadb39ec admin admin] The token cache is being invalidate because project db91e234121d481aa3a5a96233ba693d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:51.674594 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-9e5d65e1-feea-4298-a588-eb63cadb39ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:51.675392 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-9e5d65e1-feea-4298-a588-eb63cadb39ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:51.683079 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-848c3960-d943-4d1e-a81a-2efcad8660ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd9dd4a58fc844c5b2837b4bf3c2ef3b', 'request_id': 'req-848c3960-d943-4d1e-a81a-2efcad8660ce'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:51.683358 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 952/1901] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:51 2022] DELETE /identity/v3/projects/db91e234121d481aa3a5a96233ba693d => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:51.686247 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d65e1-feea-4298-a588-eb63cadb39ec admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:51.699037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-848c3960-d943-4d1e-a81a-2efcad8660ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd9dd4a58fc844c5b2837b4bf3c2ef3b', 'request_id': 'req-848c3960-d943-4d1e-a81a-2efcad8660ce'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:51.699443 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-9e5d65e1-feea-4298-a588-eb63cadb39ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:51.718113 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-848c3960-d943-4d1e-a81a-2efcad8660ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd9dd4a58fc844c5b2837b4bf3c2ef3b', 'request_id': 'req-848c3960-d943-4d1e-a81a-2efcad8660ce'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:51.754627 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-848c3960-d943-4d1e-a81a-2efcad8660ce admin admin] The token cache is being invalidate because project dd9dd4a58fc844c5b2837b4bf3c2ef3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:51.755220 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-848c3960-d943-4d1e-a81a-2efcad8660ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:51.759380 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-848c3960-d943-4d1e-a81a-2efcad8660ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:51.762048 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 950/1902] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:51 2022] DELETE /identity/v3/projects/dd9dd4a58fc844c5b2837b4bf3c2ef3b => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:51.767759 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13541943-6ca4-442e-a669-dd87dbd0e00b None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:51.769061 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13541943-6ca4-442e-a669-dd87dbd0e00b None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:51.769452 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13541943-6ca4-442e-a669-dd87dbd0e00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:51.811531 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-13541943-6ca4-442e-a669-dd87dbd0e00b None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:51.813025 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-13541943-6ca4-442e-a669-dd87dbd0e00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:51.821305 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:51.957419 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d65e1-feea-4298-a588-eb63cadb39ec admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:52.004151 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 951/1903] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:52.012045 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d4a3a-a960-4d9a-9182-9fc3f73e94af None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:52.012330 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d4a3a-a960-4d9a-9182-9fc3f73e94af None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:52.012604 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d4a3a-a960-4d9a-9182-9fc3f73e94af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:52.032467 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:52.043601 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-ae2d4a3a-a960-4d9a-9182-9fc3f73e94af None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:52.044995 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-ae2d4a3a-a960-4d9a-9182-9fc3f73e94af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:52.062952 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 952/1904] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:52.064167 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d4a3a-a960-4d9a-9182-9fc3f73e94af None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:52.074908 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d4a3a-a960-4d9a-9182-9fc3f73e94af None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:52.080546 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1a7e6-5237-4728-a995-9882bc1f8965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:52.081387 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1a7e6-5237-4728-a995-9882bc1f8965 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:52.081799 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1a7e6-5237-4728-a995-9882bc1f8965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:52.082371 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1a7e6-5237-4728-a995-9882bc1f8965 admin admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:52.083593 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c1a7e6-5237-4728-a995-9882bc1f8965 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:52.084026 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c1a7e6-5237-4728-a995-9882bc1f8965 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:52.085914 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c1a7e6-5237-4728-a995-9882bc1f8965 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:52.104364 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 953/1905] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:29:52.108735 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1a7e6-5237-4728-a995-9882bc1f8965 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:52.114195 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a64fe2-405d-4258-8921-c5c07f577371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:52.115251 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a64fe2-405d-4258-8921-c5c07f577371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:52.115535 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a64fe2-405d-4258-8921-c5c07f577371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:52.115948 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a64fe2-405d-4258-8921-c5c07f577371 admin admin] PATH_INFO: `/v3/users/b7a18dff543d4a4eba4ce673b2e533d3` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:52.117202 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a64fe2-405d-4258-8921-c5c07f577371 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7a18dff543d4a4eba4ce673b2e533d3)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:52.119578 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a64fe2-405d-4258-8921-c5c07f577371 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:52.119898 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1a7e6-5237-4728-a995-9882bc1f8965 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:52.125445 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db74553-c15b-49be-8ab1-4da081f8e688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:52.133010 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db74553-c15b-49be-8ab1-4da081f8e688 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:52.133528 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db74553-c15b-49be-8ab1-4da081f8e688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:52.133943 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db74553-c15b-49be-8ab1-4da081f8e688 admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:52.135004 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db74553-c15b-49be-8ab1-4da081f8e688 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:52.137365 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db74553-c15b-49be-8ab1-4da081f8e688 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:52.172853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 954/1906] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:52 2022] POST /identity/v3/projects => generated 387 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:29:52.177170 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db74553-c15b-49be-8ab1-4da081f8e688 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:52.204085 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-01a64fe2-405d-4258-8921-c5c07f577371 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7a18dff543d4a4eba4ce673b2e533d3'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:52.215325 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-01a64fe2-405d-4258-8921-c5c07f577371 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7a18dff543d4a4eba4ce673b2e533d3'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:52.216113 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-01a64fe2-405d-4258-8921-c5c07f577371 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7a18dff543d4a4eba4ce673b2e533d3'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:52.238563 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-01a64fe2-405d-4258-8921-c5c07f577371 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7a18dff543d4a4eba4ce673b2e533d3'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:52.250134 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db74553-c15b-49be-8ab1-4da081f8e688 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:52.253367 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-01a64fe2-405d-4258-8921-c5c07f577371 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7a18dff543d4a4eba4ce673b2e533d3'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:52.258594 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58135c8-a5a9-4d83-b605-7755f0309102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:52.259331 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58135c8-a5a9-4d83-b605-7755f0309102 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:52.259603 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58135c8-a5a9-4d83-b605-7755f0309102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:52.259939 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58135c8-a5a9-4d83-b605-7755f0309102 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:52.260628 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 953/1907] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:51 2022] DELETE /identity/v3/users/b7a18dff543d4a4eba4ce673b2e533d3 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:29:52.261191 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58135c8-a5a9-4d83-b605-7755f0309102 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:52.263430 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58135c8-a5a9-4d83-b605-7755f0309102 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:52.298922 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 955/1908] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:52 2022] POST /identity/v3/users => generated 294 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:52.299879 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a64fe2-405d-4258-8921-c5c07f577371 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:52.301422 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-01a64fe2-405d-4258-8921-c5c07f577371 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:52.410494 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:52.417397 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a64fe2-405d-4258-8921-c5c07f577371 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:52.425860 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6702e-675a-4d45-a2df-19c7ea4786b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:52.426849 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6702e-675a-4d45-a2df-19c7ea4786b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:52.427290 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6702e-675a-4d45-a2df-19c7ea4786b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:52.427705 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6702e-675a-4d45-a2df-19c7ea4786b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:52.428508 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df6702e-675a-4d45-a2df-19c7ea4786b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:52.430576 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df6702e-675a-4d45-a2df-19c7ea4786b2 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:52.459198 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58135c8-a5a9-4d83-b605-7755f0309102 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:52.465247 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 954/1909] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:52.485728 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58135c8-a5a9-4d83-b605-7755f0309102 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:52.497149 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50657a-0bf6-4a0d-b12f-aacd6372f594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:52.498334 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50657a-0bf6-4a0d-b12f-aacd6372f594 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:52.498720 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50657a-0bf6-4a0d-b12f-aacd6372f594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:52.499008 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50657a-0bf6-4a0d-b12f-aacd6372f594 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:52.499649 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb50657a-0bf6-4a0d-b12f-aacd6372f594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:52.501794 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb50657a-0bf6-4a0d-b12f-aacd6372f594 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:52.518143 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6702e-675a-4d45-a2df-19c7ea4786b2 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:52.518576 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 956/1910] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:52.529319 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6702e-675a-4d45-a2df-19c7ea4786b2 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:52.534680 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dff828-f8c1-4d1a-9168-248c21462072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:52.535363 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dff828-f8c1-4d1a-9168-248c21462072 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:52.535639 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dff828-f8c1-4d1a-9168-248c21462072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:52.535909 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dff828-f8c1-4d1a-9168-248c21462072 admin admin] PATH_INFO: `/v3/projects/32d037d6b37c475cb463096321d3bd95/users/88d2c6246ea34a27b34b5931ff0a462b/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:52.553337 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dff828-f8c1-4d1a-9168-248c21462072 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32d037d6b37c475cb463096321d3bd95, user_id=88d2c6246ea34a27b34b5931ff0a462b, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:52.557079 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dff828-f8c1-4d1a-9168-248c21462072 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:52.609162 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3049b0b-031a-4cca-8c31-921feacc64a1 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:52.609666 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3049b0b-031a-4cca-8c31-921feacc64a1 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:52.609966 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3049b0b-031a-4cca-8c31-921feacc64a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:52.613019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 955/1911] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:52 2022] PUT /identity/v3/projects/32d037d6b37c475cb463096321d3bd95/users/88d2c6246ea34a27b34b5931ff0a462b/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:52.645814 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-d3049b0b-031a-4cca-8c31-921feacc64a1 None None] MFA Rules not processed for user `88d2c6246ea34a27b34b5931ff0a462b`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:52.647016 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-d3049b0b-031a-4cca-8c31-921feacc64a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:52.755001 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dff828-f8c1-4d1a-9168-248c21462072 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:52.756238 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 957/1912] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:52.791890 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3049b0b-031a-4cca-8c31-921feacc64a1 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:52.792964 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-d3049b0b-031a-4cca-8c31-921feacc64a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:52.851990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dff828-f8c1-4d1a-9168-248c21462072 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:52.857634 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3508b3-0ba6-4491-96de-bd9b29a4c44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:52.858250 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3508b3-0ba6-4491-96de-bd9b29a4c44f admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:52.858531 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3508b3-0ba6-4491-96de-bd9b29a4c44f admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:52.858828 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3508b3-0ba6-4491-96de-bd9b29a4c44f admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:52.859514 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3508b3-0ba6-4491-96de-bd9b29a4c44f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:52.861785 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3508b3-0ba6-4491-96de-bd9b29a4c44f admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:52.867837 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:52.913471 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3049b0b-031a-4cca-8c31-921feacc64a1 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:52.923167 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a0e20-397b-468c-9b42-f34b7c012b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:52.923986 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a0e20-397b-468c-9b42-f34b7c012b58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:52.924131 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a0e20-397b-468c-9b42-f34b7c012b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:52.924341 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a0e20-397b-468c-9b42-f34b7c012b58 admin admin] PATH_INFO: `/v3/projects/ca2a5468b46a4b788de5a483ca66a3c3` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:52.925883 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a0e20-397b-468c-9b42-f34b7c012b58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca2a5468b46a4b788de5a483ca66a3c3)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:52.927852 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a0e20-397b-468c-9b42-f34b7c012b58 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:52.931646 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 956/1913] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:52 2022] POST /identity/v3/projects => generated 387 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:52.935218 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3508b3-0ba6-4491-96de-bd9b29a4c44f admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:53.050512 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3508b3-0ba6-4491-96de-bd9b29a4c44f admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:53.075326 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-085a0e20-397b-468c-9b42-f34b7c012b58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca2a5468b46a4b788de5a483ca66a3c3', 'request_id': 'req-085a0e20-397b-468c-9b42-f34b7c012b58'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:53.091749 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-085a0e20-397b-468c-9b42-f34b7c012b58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca2a5468b46a4b788de5a483ca66a3c3', 'request_id': 'req-085a0e20-397b-468c-9b42-f34b7c012b58'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:53.103913 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-085a0e20-397b-468c-9b42-f34b7c012b58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca2a5468b46a4b788de5a483ca66a3c3', 'request_id': 'req-085a0e20-397b-468c-9b42-f34b7c012b58'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:53.124716 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-085a0e20-397b-468c-9b42-f34b7c012b58 admin admin] The token cache is being invalidate because project ca2a5468b46a4b788de5a483ca66a3c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:53.125408 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-085a0e20-397b-468c-9b42-f34b7c012b58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:53.130750 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-085a0e20-397b-468c-9b42-f34b7c012b58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:53.132931 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 958/1914] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:52 2022] DELETE /identity/v3/projects/ca2a5468b46a4b788de5a483ca66a3c3 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:29:53.142011 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8eafe-31f8-4b1f-af0f-c0eb985262ca None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:53.142011 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8eafe-31f8-4b1f-af0f-c0eb985262ca None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:53.142011 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8eafe-31f8-4b1f-af0f-c0eb985262ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:53.171459 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd72e14-b0d4-42fa-879a-1c8023553101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:53.172171 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd72e14-b0d4-42fa-879a-1c8023553101 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:53.172449 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd72e14-b0d4-42fa-879a-1c8023553101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:53.172709 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd72e14-b0d4-42fa-879a-1c8023553101 admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:53.173451 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd72e14-b0d4-42fa-879a-1c8023553101 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:53.176450 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd72e14-b0d4-42fa-879a-1c8023553101 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:53.180530 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-56e8eafe-31f8-4b1f-af0f-c0eb985262ca None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:53.181969 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-56e8eafe-31f8-4b1f-af0f-c0eb985262ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:53.195043 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:53.202682 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 959/1915] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:53.212237 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765fc4b-be0d-4906-b2c4-b1fbf3c5447a None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:53.213948 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765fc4b-be0d-4906-b2c4-b1fbf3c5447a None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:53.214396 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765fc4b-be0d-4906-b2c4-b1fbf3c5447a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:53.226616 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 957/1916] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:52 2022] POST /identity/v3/users => generated 300 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:29:53.233436 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd72e14-b0d4-42fa-879a-1c8023553101 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:53.237338 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-9bd72e14-b0d4-42fa-879a-1c8023553101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:53.254828 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-4765fc4b-be0d-4906-b2c4-b1fbf3c5447a None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:53.256278 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-4765fc4b-be0d-4906-b2c4-b1fbf3c5447a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:53.268598 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd72e14-b0d4-42fa-879a-1c8023553101 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:53.270534 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 960/1917] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:53.275939 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765fc4b-be0d-4906-b2c4-b1fbf3c5447a None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:53.292342 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b4b38d-afeb-4b2e-9655-cf168c32c845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:53.292987 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4b38d-afeb-4b2e-9655-cf168c32c845 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:53.293275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4b38d-afeb-4b2e-9655-cf168c32c845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:53.293595 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4b38d-afeb-4b2e-9655-cf168c32c845 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:53.293803 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765fc4b-be0d-4906-b2c4-b1fbf3c5447a None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:53.294253 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b4b38d-afeb-4b2e-9655-cf168c32c845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:53.295999 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b4b38d-afeb-4b2e-9655-cf168c32c845 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:53.300429 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f56711-d2d6-4743-bec6-9c4a6655b123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:53.301323 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f56711-d2d6-4743-bec6-9c4a6655b123 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:53.302052 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f56711-d2d6-4743-bec6-9c4a6655b123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:53.302500 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f56711-d2d6-4743-bec6-9c4a6655b123 admin admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:53.303344 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f56711-d2d6-4743-bec6-9c4a6655b123 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:53.303810 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f56711-d2d6-4743-bec6-9c4a6655b123 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:53.306047 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f56711-d2d6-4743-bec6-9c4a6655b123 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:53.317631 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 958/1918] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:53.318773 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b4b38d-afeb-4b2e-9655-cf168c32c845 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:53.326256 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 961/1919] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:29:53.330485 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f56711-d2d6-4743-bec6-9c4a6655b123 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:53.334328 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b4b38d-afeb-4b2e-9655-cf168c32c845 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:53.340303 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe543b85-11eb-451b-89f4-02bf17d772a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:53.340675 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe543b85-11eb-451b-89f4-02bf17d772a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:53.340961 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe543b85-11eb-451b-89f4-02bf17d772a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:53.341356 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe543b85-11eb-451b-89f4-02bf17d772a9 admin admin] PATH_INFO: `/v3/projects/aa3fd99841dc4b9088cdaffd835ba6c9/users/e5f87841049f4b6dbb01e6e2c6613077/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:53.363851 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f56711-d2d6-4743-bec6-9c4a6655b123 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:53.364542 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe543b85-11eb-451b-89f4-02bf17d772a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa3fd99841dc4b9088cdaffd835ba6c9, user_id=e5f87841049f4b6dbb01e6e2c6613077, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:53.367891 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe543b85-11eb-451b-89f4-02bf17d772a9 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:53.369236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb73fdc-96aa-4a58-81e7-8320ac49480a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:53.370277 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb73fdc-96aa-4a58-81e7-8320ac49480a admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:53.370634 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb73fdc-96aa-4a58-81e7-8320ac49480a admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:53.370998 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb73fdc-96aa-4a58-81e7-8320ac49480a admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:53.371707 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb73fdc-96aa-4a58-81e7-8320ac49480a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:53.374010 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb73fdc-96aa-4a58-81e7-8320ac49480a admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:53.406646 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 959/1920] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:53 2022] PUT /identity/v3/projects/aa3fd99841dc4b9088cdaffd835ba6c9/users/e5f87841049f4b6dbb01e6e2c6613077/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:53.408107 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe543b85-11eb-451b-89f4-02bf17d772a9 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:53.413014 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 962/1921] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:53 2022] POST /identity/v3/projects => generated 373 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:53.418026 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb73fdc-96aa-4a58-81e7-8320ac49480a admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:53.489527 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe543b85-11eb-451b-89f4-02bf17d772a9 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:53.491480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb73fdc-96aa-4a58-81e7-8320ac49480a admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:53.496518 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db26a33c-ffb0-4e73-a301-cfa6d6426bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:53.497107 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26a33c-ffb0-4e73-a301-cfa6d6426bbc admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:53.497222 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512cf652-42ba-48f8-95ad-42cb9dfb20ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:53.497514 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26a33c-ffb0-4e73-a301-cfa6d6426bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:53.497853 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26a33c-ffb0-4e73-a301-cfa6d6426bbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:53.498157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512cf652-42ba-48f8-95ad-42cb9dfb20ba admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:53.498441 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512cf652-42ba-48f8-95ad-42cb9dfb20ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:53.498761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db26a33c-ffb0-4e73-a301-cfa6d6426bbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:53.498887 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512cf652-42ba-48f8-95ad-42cb9dfb20ba admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:53.499579 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512cf652-42ba-48f8-95ad-42cb9dfb20ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:53.501229 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db26a33c-ffb0-4e73-a301-cfa6d6426bbc admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:53.502088 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512cf652-42ba-48f8-95ad-42cb9dfb20ba admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:53.518264 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 960/1922] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:29:53.522715 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db26a33c-ffb0-4e73-a301-cfa6d6426bbc admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:53.531768 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 963/1923] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:53 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:53.535645 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512cf652-42ba-48f8-95ad-42cb9dfb20ba admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:53.538716 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db26a33c-ffb0-4e73-a301-cfa6d6426bbc admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:53.543475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:53.545800 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b2684-4fad-48f6-b87c-afa7f334fd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:53.546751 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b2684-4fad-48f6-b87c-afa7f334fd54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:53.547061 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b2684-4fad-48f6-b87c-afa7f334fd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:53.547424 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b2684-4fad-48f6-b87c-afa7f334fd54 admin admin] PATH_INFO: `/v3/projects/aa3fd99841dc4b9088cdaffd835ba6c9/users/e5f87841049f4b6dbb01e6e2c6613077/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:53.549489 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512cf652-42ba-48f8-95ad-42cb9dfb20ba admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:53.550172 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b2684-4fad-48f6-b87c-afa7f334fd54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa3fd99841dc4b9088cdaffd835ba6c9, user_id=e5f87841049f4b6dbb01e6e2c6613077, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:53.553780 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b2684-4fad-48f6-b87c-afa7f334fd54 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:53.559828 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d66781-ac46-4a36-95af-92c4d298e2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:53.560523 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d66781-ac46-4a36-95af-92c4d298e2de admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:53.561081 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d66781-ac46-4a36-95af-92c4d298e2de admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:53.563512 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d66781-ac46-4a36-95af-92c4d298e2de admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:53.565257 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d66781-ac46-4a36-95af-92c4d298e2de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:53.567452 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d66781-ac46-4a36-95af-92c4d298e2de admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:53.580490 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 961/1924] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:53 2022] PUT /identity/v3/projects/aa3fd99841dc4b9088cdaffd835ba6c9/users/e5f87841049f4b6dbb01e6e2c6613077/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:29:53.582158 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 964/1925] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:29:53.583014 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d66781-ac46-4a36-95af-92c4d298e2de admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:53.583421 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1930c-d5b8-493c-9355-d6479f753494 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:53.583917 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1930c-d5b8-493c-9355-d6479f753494 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:53.584478 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1930c-d5b8-493c-9355-d6479f753494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:53.594310 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d66781-ac46-4a36-95af-92c4d298e2de admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:53.631812 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-d6e1930c-d5b8-493c-9355-d6479f753494 None None] MFA Rules not processed for user `e5f87841049f4b6dbb01e6e2c6613077`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:53.633571 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-d6e1930c-d5b8-493c-9355-d6479f753494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:53.669110 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bfddcf-7068-4a72-8cbd-1c26ba0d63d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:53.669645 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bfddcf-7068-4a72-8cbd-1c26ba0d63d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:53.669921 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bfddcf-7068-4a72-8cbd-1c26ba0d63d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:53.670187 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bfddcf-7068-4a72-8cbd-1c26ba0d63d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:53.670861 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bfddcf-7068-4a72-8cbd-1c26ba0d63d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:53.672517 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bfddcf-7068-4a72-8cbd-1c26ba0d63d9 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:53.685980 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 965/1926] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:53.690452 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bfddcf-7068-4a72-8cbd-1c26ba0d63d9 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:53.701815 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bfddcf-7068-4a72-8cbd-1c26ba0d63d9 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:53.708601 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b85b8-35e8-4bc8-b60b-e8472cd8c6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:53.709218 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b85b8-35e8-4bc8-b60b-e8472cd8c6e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:53.709495 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b85b8-35e8-4bc8-b60b-e8472cd8c6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:53.709769 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b85b8-35e8-4bc8-b60b-e8472cd8c6e4 admin admin] PATH_INFO: `/v3/projects/1cf2c2a844284c4392c61565fdfc55e6/users/2b8b1a7b73be4f4e898e44caa59025c9/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:53.732839 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596b85b8-35e8-4bc8-b60b-e8472cd8c6e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cf2c2a844284c4392c61565fdfc55e6, user_id=2b8b1a7b73be4f4e898e44caa59025c9, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:53.735706 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596b85b8-35e8-4bc8-b60b-e8472cd8c6e4 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:53.774646 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 962/1928] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:53 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:53.775801 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 966/1928] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:53 2022] PUT /identity/v3/projects/1cf2c2a844284c4392c61565fdfc55e6/users/2b8b1a7b73be4f4e898e44caa59025c9/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:29:53.777358 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae828e8-43ff-4326-a610-4f828f365655 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:53.777650 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae828e8-43ff-4326-a610-4f828f365655 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:53.777929 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae828e8-43ff-4326-a610-4f828f365655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:53.787074 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b85b8-35e8-4bc8-b60b-e8472cd8c6e4 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:53.791007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-596b85b8-35e8-4bc8-b60b-e8472cd8c6e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:53.816129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-8ae828e8-43ff-4326-a610-4f828f365655 None None] MFA Rules not processed for user `2b8b1a7b73be4f4e898e44caa59025c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:53.817731 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-8ae828e8-43ff-4326-a610-4f828f365655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:53.839707 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:53.873623 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b85b8-35e8-4bc8-b60b-e8472cd8c6e4 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:53.878165 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c254a-746e-4e84-8cf2-f0997e88fa5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:53.878843 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c254a-746e-4e84-8cf2-f0997e88fa5b service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:53.879132 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c254a-746e-4e84-8cf2-f0997e88fa5b service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:53.879405 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c254a-746e-4e84-8cf2-f0997e88fa5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:53.928541 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 963/1929] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:53.940316 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae828e8-43ff-4326-a610-4f828f365655 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:53.941809 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-8ae828e8-43ff-4326-a610-4f828f365655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:53.952510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564c254a-746e-4e84-8cf2-f0997e88fa5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:53.954096 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564c254a-746e-4e84-8cf2-f0997e88fa5b service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:53.956803 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae828e8-43ff-4326-a610-4f828f365655 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:53.961213 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2e2f3-2a0f-4430-a642-af3157b3a193 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:53.961904 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2e2f3-2a0f-4430-a642-af3157b3a193 service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:53.962282 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2e2f3-2a0f-4430-a642-af3157b3a193 service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:53.962648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2e2f3-2a0f-4430-a642-af3157b3a193 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:53.978807 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2e2f3-2a0f-4430-a642-af3157b3a193 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:53.980693 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2e2f3-2a0f-4430-a642-af3157b3a193 service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:53.988062 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 967/1930] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:53 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:29:54.002530 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 964/1931] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:29:54.027874 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2e2f3-2a0f-4430-a642-af3157b3a193 service cinder] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:54.028038 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c254a-746e-4e84-8cf2-f0997e88fa5b service cinder] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:54.068989 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:54.143695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2e2f3-2a0f-4430-a642-af3157b3a193 service cinder] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:54.146895 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c254a-746e-4e84-8cf2-f0997e88fa5b service cinder] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:54.149282 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f236a-6d55-4544-ab7f-b83157e67f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:54.150141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f236a-6d55-4544-ab7f-b83157e67f02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:54.150362 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f236a-6d55-4544-ab7f-b83157e67f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:54.150683 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f236a-6d55-4544-ab7f-b83157e67f02 admin admin] PATH_INFO: `/v3/users/2b8b1a7b73be4f4e898e44caa59025c9` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:54.151943 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f236a-6d55-4544-ab7f-b83157e67f02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b8b1a7b73be4f4e898e44caa59025c9)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:54.152052 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d4818-8290-40f5-a158-8c2af62d412a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:54.152765 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d4818-8290-40f5-a158-8c2af62d412a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:54.153107 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d4818-8290-40f5-a158-8c2af62d412a admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:54.153445 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d4818-8290-40f5-a158-8c2af62d412a admin admin] PATH_INFO: `/v3/users/88d2c6246ea34a27b34b5931ff0a462b` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:54.153912 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f236a-6d55-4544-ab7f-b83157e67f02 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:54.154802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d4818-8290-40f5-a158-8c2af62d412a admin admin] RBAC: Authorizing `identity:delete_user(user_id=88d2c6246ea34a27b34b5931ff0a462b)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:54.156941 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d4818-8290-40f5-a158-8c2af62d412a admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:54.226167 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-7e7f236a-6d55-4544-ab7f-b83157e67f02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b8b1a7b73be4f4e898e44caa59025c9'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:54.226843 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a17d4818-8290-40f5-a158-8c2af62d412a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88d2c6246ea34a27b34b5931ff0a462b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:54.227622 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a17d4818-8290-40f5-a158-8c2af62d412a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88d2c6246ea34a27b34b5931ff0a462b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:54.235997 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-7e7f236a-6d55-4544-ab7f-b83157e67f02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b8b1a7b73be4f4e898e44caa59025c9'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:54.236201 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a17d4818-8290-40f5-a158-8c2af62d412a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88d2c6246ea34a27b34b5931ff0a462b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:54.236851 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-7e7f236a-6d55-4544-ab7f-b83157e67f02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b8b1a7b73be4f4e898e44caa59025c9'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:54.249665 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a17d4818-8290-40f5-a158-8c2af62d412a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88d2c6246ea34a27b34b5931ff0a462b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:54.254206 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-7e7f236a-6d55-4544-ab7f-b83157e67f02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b8b1a7b73be4f4e898e44caa59025c9'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:54.261771 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a17d4818-8290-40f5-a158-8c2af62d412a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88d2c6246ea34a27b34b5931ff0a462b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:54.265349 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-7e7f236a-6d55-4544-ab7f-b83157e67f02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b8b1a7b73be4f4e898e44caa59025c9'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:54.269033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 968/1932] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:54 2022] DELETE /identity/v3/users/88d2c6246ea34a27b34b5931ff0a462b => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:54.275690 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 965/1933] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:54 2022] DELETE /identity/v3/users/2b8b1a7b73be4f4e898e44caa59025c9 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:54.275856 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99193759-71bb-438f-8fff-9f7fb0e35759 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:54.275856 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99193759-71bb-438f-8fff-9f7fb0e35759 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:54.275856 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99193759-71bb-438f-8fff-9f7fb0e35759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:54.277269 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110aea2-93d4-4407-b1bc-dab80462be51 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:54.277758 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110aea2-93d4-4407-b1bc-dab80462be51 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:54.278096 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110aea2-93d4-4407-b1bc-dab80462be51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:54.296038 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-99193759-71bb-438f-8fff-9f7fb0e35759 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:54.297380 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-99193759-71bb-438f-8fff-9f7fb0e35759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:54.298907 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-9110aea2-93d4-4407-b1bc-dab80462be51 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:54.300255 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-9110aea2-93d4-4407-b1bc-dab80462be51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:54.318288 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:54.375661 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 969/1934] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:54.383038 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 966/1935] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:54.384288 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99193759-71bb-438f-8fff-9f7fb0e35759 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:54.391440 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-99193759-71bb-438f-8fff-9f7fb0e35759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:54.393174 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110aea2-93d4-4407-b1bc-dab80462be51 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:54.394986 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-9110aea2-93d4-4407-b1bc-dab80462be51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:54.454598 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:54.487227 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99193759-71bb-438f-8fff-9f7fb0e35759 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:54.491269 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc95bd-4f61-4ba5-a59b-79e3ee58db0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:54.492003 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc95bd-4f61-4ba5-a59b-79e3ee58db0d service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:54.492277 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc95bd-4f61-4ba5-a59b-79e3ee58db0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:54.492773 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc95bd-4f61-4ba5-a59b-79e3ee58db0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:54.498523 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110aea2-93d4-4407-b1bc-dab80462be51 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:54.502967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45864bf0-2160-4092-bd1c-6b8cee099e8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:54.503586 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45864bf0-2160-4092-bd1c-6b8cee099e8f service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:54.503870 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45864bf0-2160-4092-bd1c-6b8cee099e8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:54.504134 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45864bf0-2160-4092-bd1c-6b8cee099e8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:54.506898 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bc95bd-4f61-4ba5-a59b-79e3ee58db0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:54.508521 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bc95bd-4f61-4ba5-a59b-79e3ee58db0d service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:54.513907 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45864bf0-2160-4092-bd1c-6b8cee099e8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:54.515388 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45864bf0-2160-4092-bd1c-6b8cee099e8f service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:54.522273 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 970/1936] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:54.525975 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 967/1937] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:29:55.040158 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc95bd-4f61-4ba5-a59b-79e3ee58db0d service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:55.041606 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-47bc95bd-4f61-4ba5-a59b-79e3ee58db0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:55.054344 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45864bf0-2160-4092-bd1c-6b8cee099e8f service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:55.055819 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-45864bf0-2160-4092-bd1c-6b8cee099e8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:55.064698 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc95bd-4f61-4ba5-a59b-79e3ee58db0d service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:55.070134 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea1488-e292-40c3-814d-1ae6b163e367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:55.070812 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea1488-e292-40c3-814d-1ae6b163e367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:55.071085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea1488-e292-40c3-814d-1ae6b163e367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:55.071247 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45864bf0-2160-4092-bd1c-6b8cee099e8f service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:55.071353 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea1488-e292-40c3-814d-1ae6b163e367 admin admin] PATH_INFO: `/v3/projects/1cf2c2a844284c4392c61565fdfc55e6` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:55.072452 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea1488-e292-40c3-814d-1ae6b163e367 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cf2c2a844284c4392c61565fdfc55e6)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:55.074498 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea1488-e292-40c3-814d-1ae6b163e367 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:55.076035 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7062f136-3850-4428-b2bb-1d0531974c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:55.076669 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062f136-3850-4428-b2bb-1d0531974c10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:55.076945 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062f136-3850-4428-b2bb-1d0531974c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:55.077215 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062f136-3850-4428-b2bb-1d0531974c10 admin admin] PATH_INFO: `/v3/projects/32d037d6b37c475cb463096321d3bd95` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:55.078387 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7062f136-3850-4428-b2bb-1d0531974c10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32d037d6b37c475cb463096321d3bd95)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:55.080669 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7062f136-3850-4428-b2bb-1d0531974c10 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:55.149414 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-60ea1488-e292-40c3-814d-1ae6b163e367 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cf2c2a844284c4392c61565fdfc55e6', 'request_id': 'req-60ea1488-e292-40c3-814d-1ae6b163e367'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:55.159570 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-60ea1488-e292-40c3-814d-1ae6b163e367 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cf2c2a844284c4392c61565fdfc55e6', 'request_id': 'req-60ea1488-e292-40c3-814d-1ae6b163e367'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:55.161040 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-7062f136-3850-4428-b2bb-1d0531974c10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32d037d6b37c475cb463096321d3bd95', 'request_id': 'req-7062f136-3850-4428-b2bb-1d0531974c10'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:55.167706 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-60ea1488-e292-40c3-814d-1ae6b163e367 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cf2c2a844284c4392c61565fdfc55e6', 'request_id': 'req-60ea1488-e292-40c3-814d-1ae6b163e367'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:55.176965 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-7062f136-3850-4428-b2bb-1d0531974c10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32d037d6b37c475cb463096321d3bd95', 'request_id': 'req-7062f136-3850-4428-b2bb-1d0531974c10'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:55.190004 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-7062f136-3850-4428-b2bb-1d0531974c10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32d037d6b37c475cb463096321d3bd95', 'request_id': 'req-7062f136-3850-4428-b2bb-1d0531974c10'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:55.193254 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-60ea1488-e292-40c3-814d-1ae6b163e367 admin admin] The token cache is being invalidate because project 1cf2c2a844284c4392c61565fdfc55e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:55.193634 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-60ea1488-e292-40c3-814d-1ae6b163e367 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:55.194344 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-60ea1488-e292-40c3-814d-1ae6b163e367 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:55.199461 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 971/1938] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:55 2022] DELETE /identity/v3/projects/1cf2c2a844284c4392c61565fdfc55e6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:55.201643 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3beaf2-7832-43b6-a928-ef920fd0515d None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:55.205248 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3beaf2-7832-43b6-a928-ef920fd0515d None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:55.205873 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3beaf2-7832-43b6-a928-ef920fd0515d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:55.214951 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-7062f136-3850-4428-b2bb-1d0531974c10 admin admin] The token cache is being invalidate because project 32d037d6b37c475cb463096321d3bd95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:55.215558 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-7062f136-3850-4428-b2bb-1d0531974c10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:55.216782 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-7062f136-3850-4428-b2bb-1d0531974c10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:55.225103 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 968/1939] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:55 2022] DELETE /identity/v3/projects/32d037d6b37c475cb463096321d3bd95 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:55.227653 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7062f136-3850-4428-b2bb-1d0531974c10 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:55.229410 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-7062f136-3850-4428-b2bb-1d0531974c10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:55.233392 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-1e3beaf2-7832-43b6-a928-ef920fd0515d None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:55.267179 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-1e3beaf2-7832-43b6-a928-ef920fd0515d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:55.306013 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:55.336726 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7062f136-3850-4428-b2bb-1d0531974c10 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:55.345782 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9638699f-3c4a-44b3-b672-bda3cb1f129a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:55.346775 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638699f-3c4a-44b3-b672-bda3cb1f129a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:55.347256 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638699f-3c4a-44b3-b672-bda3cb1f129a admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:55.347696 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638699f-3c4a-44b3-b672-bda3cb1f129a admin admin] PATH_INFO: `/v3/users/e5f87841049f4b6dbb01e6e2c6613077` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:55.349780 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9638699f-3c4a-44b3-b672-bda3cb1f129a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5f87841049f4b6dbb01e6e2c6613077)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:55.352066 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9638699f-3c4a-44b3-b672-bda3cb1f129a admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:55.432802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 972/1940] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:55.437678 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33404e95-3bea-4408-b011-a45318aa01f1 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:55.437678 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33404e95-3bea-4408-b011-a45318aa01f1 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:55.438127 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33404e95-3bea-4408-b011-a45318aa01f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:55.443473 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-9638699f-3c4a-44b3-b672-bda3cb1f129a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5f87841049f4b6dbb01e6e2c6613077'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:55.456001 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-9638699f-3c4a-44b3-b672-bda3cb1f129a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5f87841049f4b6dbb01e6e2c6613077'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:55.457463 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-9638699f-3c4a-44b3-b672-bda3cb1f129a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5f87841049f4b6dbb01e6e2c6613077'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:55.475360 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-9638699f-3c4a-44b3-b672-bda3cb1f129a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5f87841049f4b6dbb01e6e2c6613077'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:55.490515 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-9638699f-3c4a-44b3-b672-bda3cb1f129a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5f87841049f4b6dbb01e6e2c6613077'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:55.518379 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-33404e95-3bea-4408-b011-a45318aa01f1 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:55.519668 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-33404e95-3bea-4408-b011-a45318aa01f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:55.525258 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 969/1941] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:55 2022] DELETE /identity/v3/users/e5f87841049f4b6dbb01e6e2c6613077 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:55.628863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 973/1942] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:55.633320 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9638699f-3c4a-44b3-b672-bda3cb1f129a admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:55.651452 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9638699f-3c4a-44b3-b672-bda3cb1f129a admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:55.666863 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633eac0a-9e9a-4307-9e21-53522dc1448e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:55.667554 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633eac0a-9e9a-4307-9e21-53522dc1448e admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:55.667844 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633eac0a-9e9a-4307-9e21-53522dc1448e admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:55.670996 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633eac0a-9e9a-4307-9e21-53522dc1448e admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:55.671774 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633eac0a-9e9a-4307-9e21-53522dc1448e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:55.672123 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633eac0a-9e9a-4307-9e21-53522dc1448e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:55.673858 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633eac0a-9e9a-4307-9e21-53522dc1448e admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:55.704104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 970/1943] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:29:55.707679 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33404e95-3bea-4408-b011-a45318aa01f1 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:55.712831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:55.720181 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33404e95-3bea-4408-b011-a45318aa01f1 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:55.735442 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c9f8c-53db-4310-b14a-bf22517a8afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:55.736186 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c9f8c-53db-4310-b14a-bf22517a8afa admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:55.737070 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c9f8c-53db-4310-b14a-bf22517a8afa admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:55.737611 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c9f8c-53db-4310-b14a-bf22517a8afa admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:55.738339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4c9f8c-53db-4310-b14a-bf22517a8afa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:55.740712 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4c9f8c-53db-4310-b14a-bf22517a8afa admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:55.847590 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 974/1944] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:55 2022] POST /identity/v3/projects => generated 373 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:55.852116 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633eac0a-9e9a-4307-9e21-53522dc1448e admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:55.884330 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633eac0a-9e9a-4307-9e21-53522dc1448e admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:56.023528 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d6676e-3d9f-44fb-b395-09d4ff58279c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:56.024264 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6676e-3d9f-44fb-b395-09d4ff58279c admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:56.024470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6676e-3d9f-44fb-b395-09d4ff58279c admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:56.024693 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6676e-3d9f-44fb-b395-09d4ff58279c admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:56.025345 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d6676e-3d9f-44fb-b395-09d4ff58279c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:56.027356 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d6676e-3d9f-44fb-b395-09d4ff58279c admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:56.071684 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 971/1945] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:55 2022] POST /identity/v3/users => generated 287 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:29:56.075716 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c9f8c-53db-4310-b14a-bf22517a8afa admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:56.108699 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d6676e-3d9f-44fb-b395-09d4ff58279c admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:56.110683 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-44d6676e-3d9f-44fb-b395-09d4ff58279c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:56.115445 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c9f8c-53db-4310-b14a-bf22517a8afa admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:56.121847 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678a5b5-41a3-4d35-b7e5-6dd5beaa09f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:56.123263 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678a5b5-41a3-4d35-b7e5-6dd5beaa09f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:56.123904 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678a5b5-41a3-4d35-b7e5-6dd5beaa09f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:56.124399 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678a5b5-41a3-4d35-b7e5-6dd5beaa09f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:56.125402 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2678a5b5-41a3-4d35-b7e5-6dd5beaa09f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:56.128534 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2678a5b5-41a3-4d35-b7e5-6dd5beaa09f3 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:56.133431 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d6676e-3d9f-44fb-b395-09d4ff58279c admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:56.149325 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 975/1946] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:56 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:56.149488 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a4680-ff5d-4e67-85ed-5ff0fa1c63a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:56.149488 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a4680-ff5d-4e67-85ed-5ff0fa1c63a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:56.149488 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a4680-ff5d-4e67-85ed-5ff0fa1c63a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:56.149488 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a4680-ff5d-4e67-85ed-5ff0fa1c63a1 admin admin] PATH_INFO: `/v3/projects/aa3fd99841dc4b9088cdaffd835ba6c9` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:56.151440 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a4680-ff5d-4e67-85ed-5ff0fa1c63a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa3fd99841dc4b9088cdaffd835ba6c9)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:56.153474 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a4680-ff5d-4e67-85ed-5ff0fa1c63a1 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:56.158551 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678a5b5-41a3-4d35-b7e5-6dd5beaa09f3 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:56.176911 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678a5b5-41a3-4d35-b7e5-6dd5beaa09f3 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:56.185839 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88720c4-59b0-4019-b858-343ef4bb9116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:56.186467 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88720c4-59b0-4019-b858-343ef4bb9116 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:56.186777 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88720c4-59b0-4019-b858-343ef4bb9116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:56.187045 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88720c4-59b0-4019-b858-343ef4bb9116 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:56.187769 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88720c4-59b0-4019-b858-343ef4bb9116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:56.192385 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88720c4-59b0-4019-b858-343ef4bb9116 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:56.220038 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 976/1947] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:56 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:56.220959 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88720c4-59b0-4019-b858-343ef4bb9116 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:56.237285 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88720c4-59b0-4019-b858-343ef4bb9116 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:56.248926 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:56.255996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58849138-3edd-4e58-9c5c-860169d0265b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:56.256692 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58849138-3edd-4e58-9c5c-860169d0265b admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:56.256967 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58849138-3edd-4e58-9c5c-860169d0265b admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:56.257238 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58849138-3edd-4e58-9c5c-860169d0265b admin admin] PATH_INFO: `/v3/projects/c60ac1d247794b90a3852e4f88c45e00/users/c585fabc2f4f4189bff19079d722e453/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:56.275702 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-cb9a4680-ff5d-4e67-85ed-5ff0fa1c63a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa3fd99841dc4b9088cdaffd835ba6c9', 'request_id': 'req-cb9a4680-ff5d-4e67-85ed-5ff0fa1c63a1'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:56.277282 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58849138-3edd-4e58-9c5c-860169d0265b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c60ac1d247794b90a3852e4f88c45e00, user_id=c585fabc2f4f4189bff19079d722e453, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:56.280058 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58849138-3edd-4e58-9c5c-860169d0265b admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:56.299711 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-cb9a4680-ff5d-4e67-85ed-5ff0fa1c63a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa3fd99841dc4b9088cdaffd835ba6c9', 'request_id': 'req-cb9a4680-ff5d-4e67-85ed-5ff0fa1c63a1'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:56.307931 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-cb9a4680-ff5d-4e67-85ed-5ff0fa1c63a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa3fd99841dc4b9088cdaffd835ba6c9', 'request_id': 'req-cb9a4680-ff5d-4e67-85ed-5ff0fa1c63a1'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:56.313333 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 977/1948] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:56 2022] PUT /identity/v3/projects/c60ac1d247794b90a3852e4f88c45e00/users/c585fabc2f4f4189bff19079d722e453/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:56.324839 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1656a76-dc49-4a86-ae4f-ce41aa7fd50d None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:56.325119 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1656a76-dc49-4a86-ae4f-ce41aa7fd50d None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:56.325390 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1656a76-dc49-4a86-ae4f-ce41aa7fd50d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:56.326519 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-cb9a4680-ff5d-4e67-85ed-5ff0fa1c63a1 admin admin] The token cache is being invalidate because project aa3fd99841dc4b9088cdaffd835ba6c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:29:56.327106 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-cb9a4680-ff5d-4e67-85ed-5ff0fa1c63a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:56.328022 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-cb9a4680-ff5d-4e67-85ed-5ff0fa1c63a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:56.336431 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 972/1949] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:56 2022] DELETE /identity/v3/projects/aa3fd99841dc4b9088cdaffd835ba6c9 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:56.353277 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299206c-8019-4ee2-b207-6e11a5cbe8da None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:56.354250 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299206c-8019-4ee2-b207-6e11a5cbe8da None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:56.354736 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299206c-8019-4ee2-b207-6e11a5cbe8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:56.369144 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-f1656a76-dc49-4a86-ae4f-ce41aa7fd50d None None] MFA Rules not processed for user `c585fabc2f4f4189bff19079d722e453`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:56.370361 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-f1656a76-dc49-4a86-ae4f-ce41aa7fd50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:56.399558 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-5299206c-8019-4ee2-b207-6e11a5cbe8da None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:56.403311 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-5299206c-8019-4ee2-b207-6e11a5cbe8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:56.483673 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 978/1950] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:56.495949 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1656a76-dc49-4a86-ae4f-ce41aa7fd50d None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:56.498154 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-f1656a76-dc49-4a86-ae4f-ce41aa7fd50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:56.542165 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 973/1951] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:56.550676 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5c45f-22bf-4f05-b7c4-3f7497637c6b None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:56.551148 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5c45f-22bf-4f05-b7c4-3f7497637c6b None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:56.551524 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5c45f-22bf-4f05-b7c4-3f7497637c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:56.614964 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-0fd5c45f-22bf-4f05-b7c4-3f7497637c6b None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:56.618609 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-0fd5c45f-22bf-4f05-b7c4-3f7497637c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:56.634689 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 974/1952] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:56.639540 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5c45f-22bf-4f05-b7c4-3f7497637c6b None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:56.641228 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1656a76-dc49-4a86-ae4f-ce41aa7fd50d None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:56.645680 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc25ab-adc0-4fb9-9374-652261a4df56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:56.646304 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc25ab-adc0-4fb9-9374-652261a4df56 service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:56.646573 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc25ab-adc0-4fb9-9374-652261a4df56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:56.646863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc25ab-adc0-4fb9-9374-652261a4df56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:56.666729 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc25ab-adc0-4fb9-9374-652261a4df56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:56.668251 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc25ab-adc0-4fb9-9374-652261a4df56 service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:56.674967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5c45f-22bf-4f05-b7c4-3f7497637c6b None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:56.686761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f55a5a-b5da-44cc-b275-72dcd7a2b747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:56.687655 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f55a5a-b5da-44cc-b275-72dcd7a2b747 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:56.688549 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f55a5a-b5da-44cc-b275-72dcd7a2b747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:56.688883 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f55a5a-b5da-44cc-b275-72dcd7a2b747 admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:56.691115 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f55a5a-b5da-44cc-b275-72dcd7a2b747 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:29:56.691712 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f55a5a-b5da-44cc-b275-72dcd7a2b747 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:56.700088 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f55a5a-b5da-44cc-b275-72dcd7a2b747 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:56.708610 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 979/1953] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:29:56.747355 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 975/1954] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:29:56.749966 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc25ab-adc0-4fb9-9374-652261a4df56 service cinder] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:56.764837 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc25ab-adc0-4fb9-9374-652261a4df56 service cinder] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:56.770647 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639dc90e-b47d-4d49-b52b-bfdb68d07456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:56.771315 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639dc90e-b47d-4d49-b52b-bfdb68d07456 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:56.771536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639dc90e-b47d-4d49-b52b-bfdb68d07456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:56.771777 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639dc90e-b47d-4d49-b52b-bfdb68d07456 admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:56.772479 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639dc90e-b47d-4d49-b52b-bfdb68d07456 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:56.774885 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639dc90e-b47d-4d49-b52b-bfdb68d07456 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:56.825063 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 980/1955] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:56 2022] POST /identity/v3/projects => generated 407 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:56.831477 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f55a5a-b5da-44cc-b275-72dcd7a2b747 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:56.848727 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f55a5a-b5da-44cc-b275-72dcd7a2b747 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:56.956570 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e43d1-2db4-495b-b742-866c4f91c145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:56.957620 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e43d1-2db4-495b-b742-866c4f91c145 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:56.958151 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e43d1-2db4-495b-b742-866c4f91c145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:56.958367 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e43d1-2db4-495b-b742-866c4f91c145 admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:56.968127 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795e43d1-2db4-495b-b742-866c4f91c145 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:56.976524 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795e43d1-2db4-495b-b742-866c4f91c145 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:57.013838 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927927a9-d575-4288-8b75-dba72a40789e None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:57.014169 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927927a9-d575-4288-8b75-dba72a40789e None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:57.014389 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927927a9-d575-4288-8b75-dba72a40789e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:57.019992 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 976/1956] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:56 2022] POST /identity/v3/users => generated 304 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:57.020854 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e43d1-2db4-495b-b742-866c4f91c145 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:57.032816 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:57.033667 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e43d1-2db4-495b-b742-866c4f91c145 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:57.040111 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ccaa51-3b1e-4072-ba1f-6e998a2c3965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:57.040803 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccaa51-3b1e-4072-ba1f-6e998a2c3965 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:57.041088 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccaa51-3b1e-4072-ba1f-6e998a2c3965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:57.041995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccaa51-3b1e-4072-ba1f-6e998a2c3965 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:57.042478 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ccaa51-3b1e-4072-ba1f-6e998a2c3965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:57.044480 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ccaa51-3b1e-4072-ba1f-6e998a2c3965 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:57.050166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-927927a9-d575-4288-8b75-dba72a40789e None None] MFA Rules not processed for user `364555ec385c4dbb91907fb9c5116d2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:57.052007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-927927a9-d575-4288-8b75-dba72a40789e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:57.063044 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 977/1957] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:29:57.064814 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ccaa51-3b1e-4072-ba1f-6e998a2c3965 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:57.085141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ccaa51-3b1e-4072-ba1f-6e998a2c3965 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:57.090539 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed69cbc-aa23-43c7-86ed-ae85dc2fc410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:57.091355 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed69cbc-aa23-43c7-86ed-ae85dc2fc410 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:57.092694 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed69cbc-aa23-43c7-86ed-ae85dc2fc410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:57.092694 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed69cbc-aa23-43c7-86ed-ae85dc2fc410 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:57.102978 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed69cbc-aa23-43c7-86ed-ae85dc2fc410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:57.104992 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed69cbc-aa23-43c7-86ed-ae85dc2fc410 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:57.131036 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 978/1958] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:29:57.135873 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed69cbc-aa23-43c7-86ed-ae85dc2fc410 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:57.151643 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed69cbc-aa23-43c7-86ed-ae85dc2fc410 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:57.157780 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 981/1959] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Nov 2 07:29:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:57.158610 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab565b12-7363-41f7-9ed3-eed576467654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:57.159611 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab565b12-7363-41f7-9ed3-eed576467654 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:57.160089 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab565b12-7363-41f7-9ed3-eed576467654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:57.160587 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab565b12-7363-41f7-9ed3-eed576467654 admin admin] PATH_INFO: `/v3/projects/8fe580d37fe7404b8f386f66e0e96c72/users/30d8e7e1500f4e98935f3dd366488b0b/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:57.173706 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927927a9-d575-4288-8b75-dba72a40789e None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:57.178004 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-927927a9-d575-4288-8b75-dba72a40789e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:57.182093 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:57.209669 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab565b12-7363-41f7-9ed3-eed576467654 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fe580d37fe7404b8f386f66e0e96c72, user_id=30d8e7e1500f4e98935f3dd366488b0b, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:57.216182 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab565b12-7363-41f7-9ed3-eed576467654 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:57.256082 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 979/1960] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:57 2022] PUT /identity/v3/projects/8fe580d37fe7404b8f386f66e0e96c72/users/30d8e7e1500f4e98935f3dd366488b0b/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:57.266918 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ec58a-41fc-47a3-969f-ccd1d316d783 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:57.267196 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ec58a-41fc-47a3-969f-ccd1d316d783 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:57.267473 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ec58a-41fc-47a3-969f-ccd1d316d783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:57.326399 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-992ec58a-41fc-47a3-969f-ccd1d316d783 None None] MFA Rules not processed for user `30d8e7e1500f4e98935f3dd366488b0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:57.328303 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-992ec58a-41fc-47a3-969f-ccd1d316d783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:57.458191 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927927a9-d575-4288-8b75-dba72a40789e None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:57.537826 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 980/1961] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:57 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:29:57.540816 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ec58a-41fc-47a3-969f-ccd1d316d783 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:57.643565 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e11a99-9541-4a00-bf4c-429679b7ac80 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '836c4044aaf74de391219289b57d2956', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:57.644338 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e11a99-9541-4a00-bf4c-429679b7ac80 service swift] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:57.644715 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e11a99-9541-4a00-bf4c-429679b7ac80 service swift] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:57.645173 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e11a99-9541-4a00-bf4c-429679b7ac80 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:57.722333 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ec58a-41fc-47a3-969f-ccd1d316d783 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:57.737070 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:57.743555 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057996f-d316-46d0-a5ce-48d468e27c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:57.744421 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057996f-d316-46d0-a5ce-48d468e27c78 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:57.744535 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057996f-d316-46d0-a5ce-48d468e27c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:57.744782 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057996f-d316-46d0-a5ce-48d468e27c78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:57.747227 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e11a99-9541-4a00-bf4c-429679b7ac80 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:57.750325 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2057996f-d316-46d0-a5ce-48d468e27c78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:57.752798 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e11a99-9541-4a00-bf4c-429679b7ac80 service swift] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:57.753948 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2057996f-d316-46d0-a5ce-48d468e27c78 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:57.790044 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 982/1962] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Nov 2 07:29:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:29:57.793547 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e11a99-9541-4a00-bf4c-429679b7ac80 service swift] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:57.796515 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-a0e11a99-9541-4a00-bf4c-429679b7ac80 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:57.820820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 981/1963] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:29:57 2022] POST /identity/v3/projects => generated 407 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:57.825183 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057996f-d316-46d0-a5ce-48d468e27c78 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:57.826544 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e11a99-9541-4a00-bf4c-429679b7ac80 service swift] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:57.913105 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057996f-d316-46d0-a5ce-48d468e27c78 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:57.924600 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ec247-a75f-49f7-9058-6f3194748e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:57.926880 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ec247-a75f-49f7-9058-6f3194748e79 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:57.927608 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ec247-a75f-49f7-9058-6f3194748e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:57.929258 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ec247-a75f-49f7-9058-6f3194748e79 admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:57.931159 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ec247-a75f-49f7-9058-6f3194748e79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:57.935565 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ec247-a75f-49f7-9058-6f3194748e79 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:57.948275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:57.980475 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 982/1964] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:29:57 2022] POST /identity/v3/users => generated 310 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:29:57.984418 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ec247-a75f-49f7-9058-6f3194748e79 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:58.004242 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ec247-a75f-49f7-9058-6f3194748e79 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:58.012572 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a59052-04ed-4451-9da4-9a7ecad9376f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:58.013214 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a59052-04ed-4451-9da4-9a7ecad9376f admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:58.013507 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a59052-04ed-4451-9da4-9a7ecad9376f admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:58.013757 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a59052-04ed-4451-9da4-9a7ecad9376f admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:58.014976 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a59052-04ed-4451-9da4-9a7ecad9376f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:58.016332 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a59052-04ed-4451-9da4-9a7ecad9376f admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:58.038182 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 983/1965] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:29:58.043322 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a59052-04ed-4451-9da4-9a7ecad9376f admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:58.044620 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda015f-f227-4b89-b275-bf9f143dcd9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:58.045593 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda015f-f227-4b89-b275-bf9f143dcd9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:58.047016 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda015f-f227-4b89-b275-bf9f143dcd9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:58.047804 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda015f-f227-4b89-b275-bf9f143dcd9e admin admin] PATH_INFO: `/v3/users/1063cf561f854909ac25c8d96b5009a4` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:58.051979 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bda015f-f227-4b89-b275-bf9f143dcd9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1063cf561f854909ac25c8d96b5009a4)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:58.057015 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bda015f-f227-4b89-b275-bf9f143dcd9e admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:58.060872 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a59052-04ed-4451-9da4-9a7ecad9376f admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:58.075432 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe4f69-74f8-4673-90d1-660caf63d47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:58.076287 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe4f69-74f8-4673-90d1-660caf63d47c admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:58.076560 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe4f69-74f8-4673-90d1-660caf63d47c admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:58.076828 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe4f69-74f8-4673-90d1-660caf63d47c admin admin] PATH_INFO: `/v3/projects/b35c6964108140d69f68ed4d9d7a37d0/users/c06ec777b52448d3bf4f1952a7b7b65b/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:58.108707 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfe4f69-74f8-4673-90d1-660caf63d47c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b35c6964108140d69f68ed4d9d7a37d0, user_id=c06ec777b52448d3bf4f1952a7b7b65b, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:58.111794 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfe4f69-74f8-4673-90d1-660caf63d47c admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:58.135441 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 984/1966] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:58 2022] PUT /identity/v3/projects/b35c6964108140d69f68ed4d9d7a37d0/users/c06ec777b52448d3bf4f1952a7b7b65b/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:58.139426 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe4f69-74f8-4673-90d1-660caf63d47c admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:58.181310 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-4bda015f-f227-4b89-b275-bf9f143dcd9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1063cf561f854909ac25c8d96b5009a4'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:58.182526 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-4bda015f-f227-4b89-b275-bf9f143dcd9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1063cf561f854909ac25c8d96b5009a4'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:58.192698 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-4bda015f-f227-4b89-b275-bf9f143dcd9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1063cf561f854909ac25c8d96b5009a4'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:58.221145 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-4bda015f-f227-4b89-b275-bf9f143dcd9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1063cf561f854909ac25c8d96b5009a4'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:58.242079 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-4bda015f-f227-4b89-b275-bf9f143dcd9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1063cf561f854909ac25c8d96b5009a4'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:29:58.263819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 983/1967] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:29:57 2022] DELETE /identity/v3/users/1063cf561f854909ac25c8d96b5009a4 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:58.268657 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd5ddb-9743-45f3-b2dc-3342e53c4510 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:58.269345 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd5ddb-9743-45f3-b2dc-3342e53c4510 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:58.269768 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd5ddb-9743-45f3-b2dc-3342e53c4510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:58.345322 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-48cd5ddb-9743-45f3-b2dc-3342e53c4510 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:58.348422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-48cd5ddb-9743-45f3-b2dc-3342e53c4510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:58.422835 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe4f69-74f8-4673-90d1-660caf63d47c admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:58.482293 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 984/1968] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:58.492910 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd5ddb-9743-45f3-b2dc-3342e53c4510 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:58.497982 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-48cd5ddb-9743-45f3-b2dc-3342e53c4510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:58.559683 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2950d6a-fc48-407a-b00c-54cd5c01f357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:58.561545 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2950d6a-fc48-407a-b00c-54cd5c01f357 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:58.561944 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2950d6a-fc48-407a-b00c-54cd5c01f357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:58.563198 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:58.563390 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2950d6a-fc48-407a-b00c-54cd5c01f357 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:58.564239 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2950d6a-fc48-407a-b00c-54cd5c01f357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:58.573220 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2950d6a-fc48-407a-b00c-54cd5c01f357 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:58.659219 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 985/1969] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 520 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:29:58.661739 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2950d6a-fc48-407a-b00c-54cd5c01f357 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:58.666111 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-f2950d6a-fc48-407a-b00c-54cd5c01f357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:58.706226 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2950d6a-fc48-407a-b00c-54cd5c01f357 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:58.734874 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e517d-758b-41b1-8a28-84abb905ef5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:58.735587 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e517d-758b-41b1-8a28-84abb905ef5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:58.735863 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e517d-758b-41b1-8a28-84abb905ef5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:58.736131 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e517d-758b-41b1-8a28-84abb905ef5d admin admin] PATH_INFO: `/v3/projects/b35c6964108140d69f68ed4d9d7a37d0/users/c06ec777b52448d3bf4f1952a7b7b65b/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:58.768133 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e517d-758b-41b1-8a28-84abb905ef5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b35c6964108140d69f68ed4d9d7a37d0, user_id=c06ec777b52448d3bf4f1952a7b7b65b, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:58.779006 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e517d-758b-41b1-8a28-84abb905ef5d admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:58.793303 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd5ddb-9743-45f3-b2dc-3342e53c4510 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:58.810393 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc84de2-cae3-4aad-8aa5-70740a47a971 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:58.811219 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc84de2-cae3-4aad-8aa5-70740a47a971 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:58.811493 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc84de2-cae3-4aad-8aa5-70740a47a971 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:58.811758 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc84de2-cae3-4aad-8aa5-70740a47a971 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:58.823676 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 986/1970] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:29:58 2022] PUT /identity/v3/projects/b35c6964108140d69f68ed4d9d7a37d0/users/c06ec777b52448d3bf4f1952a7b7b65b/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:29:58.850453 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e58084-8887-493f-b64b-86da33f864e7 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:58.851546 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc84de2-cae3-4aad-8aa5-70740a47a971 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:58.853121 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e58084-8887-493f-b64b-86da33f864e7 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:58.853252 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc84de2-cae3-4aad-8aa5-70740a47a971 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:58.853411 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e58084-8887-493f-b64b-86da33f864e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:58.895394 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 985/1971] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:59.032174 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-92e58084-8887-493f-b64b-86da33f864e7 None None] MFA Rules not processed for user `c06ec777b52448d3bf4f1952a7b7b65b`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:29:59.033456 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-92e58084-8887-493f-b64b-86da33f864e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:59.208189 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:29:59.394849 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 987/1972] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:29:58 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 549 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:29:59.432414 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc84de2-cae3-4aad-8aa5-70740a47a971 service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:59.434462 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-7fc84de2-cae3-4aad-8aa5-70740a47a971 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:59.686110 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc84de2-cae3-4aad-8aa5-70740a47a971 service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:29:59.714821 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e58084-8887-493f-b64b-86da33f864e7 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:29:59.716622 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-92e58084-8887-493f-b64b-86da33f864e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:29:59.856423 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6a838-4193-4114-8bf1-59b050cb378d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:29:59.857057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6a838-4193-4114-8bf1-59b050cb378d service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:29:59.857332 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6a838-4193-4114-8bf1-59b050cb378d service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:29:59.857599 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6a838-4193-4114-8bf1-59b050cb378d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:29:59.883093 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd6a838-4193-4114-8bf1-59b050cb378d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:29:59.885074 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd6a838-4193-4114-8bf1-59b050cb378d service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:29:59.911035 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 986/1973] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:29:59 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:29:59.996499 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e58084-8887-493f-b64b-86da33f864e7 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:00.021487 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d514815-d627-4108-9896-0b62f82ca00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:00.027949 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d514815-d627-4108-9896-0b62f82ca00c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:00.028390 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d514815-d627-4108-9896-0b62f82ca00c admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:00.028751 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d514815-d627-4108-9896-0b62f82ca00c admin admin] PATH_INFO: `/v3/projects/df987d9c213649969f9c92c14fb39d90` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:00.038700 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d514815-d627-4108-9896-0b62f82ca00c admin admin] RBAC: Authorizing `identity:delete_project(project_id=df987d9c213649969f9c92c14fb39d90)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:00.041115 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d514815-d627-4108-9896-0b62f82ca00c admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:00.302103 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-6d514815-d627-4108-9896-0b62f82ca00c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df987d9c213649969f9c92c14fb39d90', 'request_id': 'req-6d514815-d627-4108-9896-0b62f82ca00c'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:00.328399 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-6d514815-d627-4108-9896-0b62f82ca00c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df987d9c213649969f9c92c14fb39d90', 'request_id': 'req-6d514815-d627-4108-9896-0b62f82ca00c'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:00.344525 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-6d514815-d627-4108-9896-0b62f82ca00c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df987d9c213649969f9c92c14fb39d90', 'request_id': 'req-6d514815-d627-4108-9896-0b62f82ca00c'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:00.383671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-6d514815-d627-4108-9896-0b62f82ca00c admin admin] The token cache is being invalidate because project df987d9c213649969f9c92c14fb39d90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:00.388777 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-6d514815-d627-4108-9896-0b62f82ca00c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:00.403651 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-6d514815-d627-4108-9896-0b62f82ca00c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:00.410424 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 988/1974] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:29:59 2022] DELETE /identity/v3/projects/df987d9c213649969f9c92c14fb39d90 => generated 0 bytes in 697 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:00.419936 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6aff8-636f-454d-ac93-ae2b7f6eba1e None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:00.420713 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6aff8-636f-454d-ac93-ae2b7f6eba1e None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:00.421151 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6aff8-636f-454d-ac93-ae2b7f6eba1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:00.484820 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-a5f6aff8-636f-454d-ac93-ae2b7f6eba1e None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:00.486846 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-a5f6aff8-636f-454d-ac93-ae2b7f6eba1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:00.588717 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c8920-5c32-4efe-ae63-651c36237730 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:00.589267 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c8920-5c32-4efe-ae63-651c36237730 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:00.589652 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c8920-5c32-4efe-ae63-651c36237730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:00.658907 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-d89c8920-5c32-4efe-ae63-651c36237730 None None] MFA Rules not processed for user `a47b81144eba4283aac610c600e73ed3`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:00.664316 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-d89c8920-5c32-4efe-ae63-651c36237730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:00.784504 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 987/1975] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:00.791165 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f6aff8-636f-454d-ac93-ae2b7f6eba1e None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:00.791165 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-a5f6aff8-636f-454d-ac93-ae2b7f6eba1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:00.829353 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:00.850333 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:00.867932 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 989/1976] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:30:00.882567 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02455a2-421c-431e-8fb1-c1980a3498fd None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:00.883273 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02455a2-421c-431e-8fb1-c1980a3498fd None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:00.883818 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02455a2-421c-431e-8fb1-c1980a3498fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:00.954640 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-b02455a2-421c-431e-8fb1-c1980a3498fd None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:00.959521 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f6aff8-636f-454d-ac93-ae2b7f6eba1e None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:00.961445 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-b02455a2-421c-431e-8fb1-c1980a3498fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:00.971237 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc1245-6bb7-479f-a6f9-a407978b507b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:00.971877 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc1245-6bb7-479f-a6f9-a407978b507b service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:00.972203 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc1245-6bb7-479f-a6f9-a407978b507b service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:00.972475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc1245-6bb7-479f-a6f9-a407978b507b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:00.974351 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-cefc1245-6bb7-479f-a6f9-a407978b507b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:00.989932 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 990/1977] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:00.989932 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02455a2-421c-431e-8fb1-c1980a3498fd None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:00.990751 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-b02455a2-421c-431e-8fb1-c1980a3498fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:01.032544 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02455a2-421c-431e-8fb1-c1980a3498fd None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:01.048939 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f554c43-24e9-4e50-8c9a-37f747f3141c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:01.062807 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f554c43-24e9-4e50-8c9a-37f747f3141c admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:01.062807 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f554c43-24e9-4e50-8c9a-37f747f3141c admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:01.062807 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f554c43-24e9-4e50-8c9a-37f747f3141c admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:01.062807 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f554c43-24e9-4e50-8c9a-37f747f3141c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:01.062807 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f554c43-24e9-4e50-8c9a-37f747f3141c admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:01.127866 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefc1245-6bb7-479f-a6f9-a407978b507b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:01.131015 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefc1245-6bb7-479f-a6f9-a407978b507b service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:01.134493 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 991/1978] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:00 2022] POST /identity/v3/projects => generated 377 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:01.135229 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f554c43-24e9-4e50-8c9a-37f747f3141c admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:01.156737 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 988/1979] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:00 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:01.157628 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc1245-6bb7-479f-a6f9-a407978b507b service cinder] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:01.184595 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc1245-6bb7-479f-a6f9-a407978b507b service cinder] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:01.278264 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f554c43-24e9-4e50-8c9a-37f747f3141c admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:01.307037 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da4515b-223e-45b4-9b25-e627b9a8e775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:01.307897 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da4515b-223e-45b4-9b25-e627b9a8e775 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:01.308107 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da4515b-223e-45b4-9b25-e627b9a8e775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:01.308216 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da4515b-223e-45b4-9b25-e627b9a8e775 admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:01.308824 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da4515b-223e-45b4-9b25-e627b9a8e775 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:30:01.309122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da4515b-223e-45b4-9b25-e627b9a8e775 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:01.310772 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da4515b-223e-45b4-9b25-e627b9a8e775 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:01.381943 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ed130e-5047-42a4-bd5d-b2f708751007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:01.383070 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed130e-5047-42a4-bd5d-b2f708751007 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:01.392134 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 992/1980] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:30:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:30:01.396108 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed130e-5047-42a4-bd5d-b2f708751007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:01.396468 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed130e-5047-42a4-bd5d-b2f708751007 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:01.397892 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da4515b-223e-45b4-9b25-e627b9a8e775 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:01.413486 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da4515b-223e-45b4-9b25-e627b9a8e775 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:01.415240 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ed130e-5047-42a4-bd5d-b2f708751007 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:01.422793 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b7e71a-9741-4572-b7f2-815aa5cbc370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:01.423698 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b7e71a-9741-4572-b7f2-815aa5cbc370 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:01.424412 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b7e71a-9741-4572-b7f2-815aa5cbc370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:01.424834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b7e71a-9741-4572-b7f2-815aa5cbc370 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:01.425465 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b7e71a-9741-4572-b7f2-815aa5cbc370 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:01.427699 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b7e71a-9741-4572-b7f2-815aa5cbc370 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:01.429032 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ed130e-5047-42a4-bd5d-b2f708751007 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:01.492312 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 993/1981] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:01 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:30:01.498191 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b7e71a-9741-4572-b7f2-815aa5cbc370 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:01.516260 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 989/1982] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:01 2022] POST /identity/v3/users => generated 295 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:01.518818 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ed130e-5047-42a4-bd5d-b2f708751007 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:01.562344 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ed130e-5047-42a4-bd5d-b2f708751007 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:01.658901 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b7e71a-9741-4572-b7f2-815aa5cbc370 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:01.669850 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bdad5-f247-4257-9300-6f9f602eaa99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:01.670982 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bdad5-f247-4257-9300-6f9f602eaa99 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:01.671665 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bdad5-f247-4257-9300-6f9f602eaa99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:01.672165 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bdad5-f247-4257-9300-6f9f602eaa99 admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:01.678329 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310bdad5-f247-4257-9300-6f9f602eaa99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:01.685382 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310bdad5-f247-4257-9300-6f9f602eaa99 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:01.749005 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 994/1983] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:01 2022] POST /identity/v3/users => generated 292 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:01.753202 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bdad5-f247-4257-9300-6f9f602eaa99 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:01.761773 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b50930-71d4-4952-a69b-ade761833107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:01.768424 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b50930-71d4-4952-a69b-ade761833107 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:01.768821 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b50930-71d4-4952-a69b-ade761833107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:01.769206 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b50930-71d4-4952-a69b-ade761833107 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:01.771013 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b50930-71d4-4952-a69b-ade761833107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:01.775696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b50930-71d4-4952-a69b-ade761833107 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:01.776720 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bdad5-f247-4257-9300-6f9f602eaa99 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:01.777916 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:01.817539 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 990/1984] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:01.818241 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86660e-4c92-4b61-8b3c-9e08ea5973c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:01.818241 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86660e-4c92-4b61-8b3c-9e08ea5973c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:01.819618 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b50930-71d4-4952-a69b-ade761833107 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:01.823649 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86660e-4c92-4b61-8b3c-9e08ea5973c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:01.824048 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86660e-4c92-4b61-8b3c-9e08ea5973c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:01.826114 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f86660e-4c92-4b61-8b3c-9e08ea5973c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:01.833858 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b50930-71d4-4952-a69b-ade761833107 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:01.839041 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f86660e-4c92-4b61-8b3c-9e08ea5973c5 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:01.860307 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c11247-cd10-43e9-b8eb-2f611f4858a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:01.864305 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c11247-cd10-43e9-b8eb-2f611f4858a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:01.866690 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c11247-cd10-43e9-b8eb-2f611f4858a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:01.867889 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c11247-cd10-43e9-b8eb-2f611f4858a0 admin admin] PATH_INFO: `/v3/projects/e4253bbcacfe4775940efe5160a90309/users/d101c78015b842f99ea0662f60aa7283/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:01.893280 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 995/1985] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:01.896054 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86660e-4c92-4b61-8b3c-9e08ea5973c5 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:01.924661 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86660e-4c92-4b61-8b3c-9e08ea5973c5 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:01.942221 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e250cde1-8d67-40ba-aac5-e83b5faea9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:01.942943 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250cde1-8d67-40ba-aac5-e83b5faea9c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:01.943218 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250cde1-8d67-40ba-aac5-e83b5faea9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:01.943485 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250cde1-8d67-40ba-aac5-e83b5faea9c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:01.945226 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e250cde1-8d67-40ba-aac5-e83b5faea9c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:01.949748 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e250cde1-8d67-40ba-aac5-e83b5faea9c9 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:01.985614 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 996/1986] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:01.988600 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e250cde1-8d67-40ba-aac5-e83b5faea9c9 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:01.988750 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c11247-cd10-43e9-b8eb-2f611f4858a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4253bbcacfe4775940efe5160a90309, user_id=d101c78015b842f99ea0662f60aa7283, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:02.002706 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c11247-cd10-43e9-b8eb-2f611f4858a0 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:02.008008 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:02.042900 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e250cde1-8d67-40ba-aac5-e83b5faea9c9 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:02.043436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 991/1987] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:01 2022] PUT /identity/v3/projects/e4253bbcacfe4775940efe5160a90309/users/d101c78015b842f99ea0662f60aa7283/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:30:02.044848 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c11247-cd10-43e9-b8eb-2f611f4858a0 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:02.198933 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c11247-cd10-43e9-b8eb-2f611f4858a0 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:02.224555 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0169823-e7ab-4bda-b828-85725669645e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:02.225201 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0169823-e7ab-4bda-b828-85725669645e admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:02.225544 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0169823-e7ab-4bda-b828-85725669645e admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:02.225766 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0169823-e7ab-4bda-b828-85725669645e admin admin] PATH_INFO: `/v3/projects/7f5f2149dcc54fa696e2dfc60f0c6dd0/users/0a6c7f817c564e9ebc6e313d43d5325c/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:02.247221 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017db66-9482-431c-a9da-88d4f6b376ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:02.248867 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017db66-9482-431c-a9da-88d4f6b376ff admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:02.249073 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017db66-9482-431c-a9da-88d4f6b376ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:02.249352 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017db66-9482-431c-a9da-88d4f6b376ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:02.250305 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017db66-9482-431c-a9da-88d4f6b376ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:02.252171 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017db66-9482-431c-a9da-88d4f6b376ff admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:02.268981 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0169823-e7ab-4bda-b828-85725669645e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f5f2149dcc54fa696e2dfc60f0c6dd0, user_id=0a6c7f817c564e9ebc6e313d43d5325c, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:02.273689 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0169823-e7ab-4bda-b828-85725669645e admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:02.302565 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 992/1988] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:02.303542 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017db66-9482-431c-a9da-88d4f6b376ff admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:02.329524 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 997/1989] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:01 2022] PUT /identity/v3/projects/7f5f2149dcc54fa696e2dfc60f0c6dd0/users/0a6c7f817c564e9ebc6e313d43d5325c/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:02.337171 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017db66-9482-431c-a9da-88d4f6b376ff admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:02.348706 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581997a7-593d-4da1-bdea-3e6e8526476a None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:02.349204 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581997a7-593d-4da1-bdea-3e6e8526476a None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:02.349658 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581997a7-593d-4da1-bdea-3e6e8526476a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:02.405687 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-581997a7-593d-4da1-bdea-3e6e8526476a None None] MFA Rules not processed for user `0a6c7f817c564e9ebc6e313d43d5325c`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:02.407720 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-581997a7-593d-4da1-bdea-3e6e8526476a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:02.508139 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306a75c-6d01-4a6d-966f-7b6c17ca6d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:02.508831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306a75c-6d01-4a6d-966f-7b6c17ca6d0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:02.509108 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306a75c-6d01-4a6d-966f-7b6c17ca6d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:02.509378 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306a75c-6d01-4a6d-966f-7b6c17ca6d0e admin admin] PATH_INFO: `/v3/projects/e4253bbcacfe4775940efe5160a90309/users/d101c78015b842f99ea0662f60aa7283/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:02.511714 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6306a75c-6d01-4a6d-966f-7b6c17ca6d0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4253bbcacfe4775940efe5160a90309, user_id=d101c78015b842f99ea0662f60aa7283, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:02.514381 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6306a75c-6d01-4a6d-966f-7b6c17ca6d0e admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:02.540333 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 993/1990] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:02 2022] PUT /identity/v3/projects/e4253bbcacfe4775940efe5160a90309/users/d101c78015b842f99ea0662f60aa7283/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:02.640712 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 998/1991] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:30:02.654759 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306a75c-6d01-4a6d-966f-7b6c17ca6d0e admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:02.791183 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306a75c-6d01-4a6d-966f-7b6c17ca6d0e admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:02.796085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee32fe-f2de-484e-9482-f4d87ad6ce6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:02.796745 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee32fe-f2de-484e-9482-f4d87ad6ce6d service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:02.797047 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee32fe-f2de-484e-9482-f4d87ad6ce6d service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:02.797327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee32fe-f2de-484e-9482-f4d87ad6ce6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:02.817194 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:02.877867 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ee32fe-f2de-484e-9482-f4d87ad6ce6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:02.883244 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ee32fe-f2de-484e-9482-f4d87ad6ce6d service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:02.901463 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 994/1992] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:05.780408 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581997a7-593d-4da1-bdea-3e6e8526476a None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:05.786508 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-581997a7-593d-4da1-bdea-3e6e8526476a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:05.859352 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581997a7-593d-4da1-bdea-3e6e8526476a None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:06.114315 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c686395-b219-4442-a0d3-24ef7286166c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:06.115388 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c686395-b219-4442-a0d3-24ef7286166c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:06.120432 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c686395-b219-4442-a0d3-24ef7286166c admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:06.121607 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c686395-b219-4442-a0d3-24ef7286166c admin admin] PATH_INFO: `/v3/users/c585fabc2f4f4189bff19079d722e453` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:06.128250 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c686395-b219-4442-a0d3-24ef7286166c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c585fabc2f4f4189bff19079d722e453)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:06.134350 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c686395-b219-4442-a0d3-24ef7286166c admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:06.205341 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:06.316035 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5c686395-b219-4442-a0d3-24ef7286166c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c585fabc2f4f4189bff19079d722e453'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:06.335802 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5c686395-b219-4442-a0d3-24ef7286166c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c585fabc2f4f4189bff19079d722e453'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:06.340344 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5c686395-b219-4442-a0d3-24ef7286166c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c585fabc2f4f4189bff19079d722e453'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:06.395380 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5c686395-b219-4442-a0d3-24ef7286166c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c585fabc2f4f4189bff19079d722e453'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:06.414863 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5c686395-b219-4442-a0d3-24ef7286166c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c585fabc2f4f4189bff19079d722e453'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:06.434502 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 999/1993] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:05 2022] DELETE /identity/v3/users/c585fabc2f4f4189bff19079d722e453 => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:06.436510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c9be1-8585-4293-935f-8efa7ee2d9a3 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:06.437064 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c9be1-8585-4293-935f-8efa7ee2d9a3 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:06.437904 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c9be1-8585-4293-935f-8efa7ee2d9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:06.527441 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-6a1c9be1-8585-4293-935f-8efa7ee2d9a3 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:06.539099 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-6a1c9be1-8585-4293-935f-8efa7ee2d9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:06.764446 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 995/1994] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:06.822377 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c686395-b219-4442-a0d3-24ef7286166c admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:06.824225 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-5c686395-b219-4442-a0d3-24ef7286166c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:06.965409 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c686395-b219-4442-a0d3-24ef7286166c admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:06.977517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cceeb5a-0060-4c45-809f-b58b768babfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:06.978159 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cceeb5a-0060-4c45-809f-b58b768babfd service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:06.978499 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cceeb5a-0060-4c45-809f-b58b768babfd service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:06.978790 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cceeb5a-0060-4c45-809f-b58b768babfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:07.026682 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cceeb5a-0060-4c45-809f-b58b768babfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:07.028709 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cceeb5a-0060-4c45-809f-b58b768babfd service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:07.052652 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1000/1995] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:07.349094 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c9be1-8585-4293-935f-8efa7ee2d9a3 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:07.352176 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-6a1c9be1-8585-4293-935f-8efa7ee2d9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:07.367711 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c9be1-8585-4293-935f-8efa7ee2d9a3 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:07.374205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973f13d-f148-435e-96c2-e4b0d342b9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:07.375403 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973f13d-f148-435e-96c2-e4b0d342b9b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:07.375856 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973f13d-f148-435e-96c2-e4b0d342b9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:07.381983 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973f13d-f148-435e-96c2-e4b0d342b9b4 admin admin] PATH_INFO: `/v3/projects/c60ac1d247794b90a3852e4f88c45e00` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:07.384235 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973f13d-f148-435e-96c2-e4b0d342b9b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c60ac1d247794b90a3852e4f88c45e00)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:07.386843 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973f13d-f148-435e-96c2-e4b0d342b9b4 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:07.473146 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-4973f13d-f148-435e-96c2-e4b0d342b9b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c60ac1d247794b90a3852e4f88c45e00', 'request_id': 'req-4973f13d-f148-435e-96c2-e4b0d342b9b4'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:07.484153 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-4973f13d-f148-435e-96c2-e4b0d342b9b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c60ac1d247794b90a3852e4f88c45e00', 'request_id': 'req-4973f13d-f148-435e-96c2-e4b0d342b9b4'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:07.495536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-4973f13d-f148-435e-96c2-e4b0d342b9b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c60ac1d247794b90a3852e4f88c45e00', 'request_id': 'req-4973f13d-f148-435e-96c2-e4b0d342b9b4'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:07.533858 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-4973f13d-f148-435e-96c2-e4b0d342b9b4 admin admin] The token cache is being invalidate because project c60ac1d247794b90a3852e4f88c45e00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:07.534644 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-4973f13d-f148-435e-96c2-e4b0d342b9b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:07.535729 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-4973f13d-f148-435e-96c2-e4b0d342b9b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:07.538148 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 996/1996] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:07 2022] DELETE /identity/v3/projects/c60ac1d247794b90a3852e4f88c45e00 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:07.544442 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7919d83-5e47-4a3f-b62d-5dc4003441b6 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:07.544788 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7919d83-5e47-4a3f-b62d-5dc4003441b6 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:07.545275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7919d83-5e47-4a3f-b62d-5dc4003441b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:07.582203 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-f7919d83-5e47-4a3f-b62d-5dc4003441b6 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:07.583873 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-f7919d83-5e47-4a3f-b62d-5dc4003441b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:07.682695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1001/1997] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:07.692550 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c1b67-0813-4042-ad07-00d884d30a53 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:07.693207 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c1b67-0813-4042-ad07-00d884d30a53 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:07.693632 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c1b67-0813-4042-ad07-00d884d30a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:07.720943 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-381c1b67-0813-4042-ad07-00d884d30a53 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:07.723617 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-381c1b67-0813-4042-ad07-00d884d30a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:07.750521 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 997/1998] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:07.750671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7919d83-5e47-4a3f-b62d-5dc4003441b6 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:07.767414 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7919d83-5e47-4a3f-b62d-5dc4003441b6 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:07.781657 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e99a8-7ee0-42bb-9365-b3f9006fe560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:07.782335 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e99a8-7ee0-42bb-9365-b3f9006fe560 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:07.782754 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e99a8-7ee0-42bb-9365-b3f9006fe560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:07.783191 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e99a8-7ee0-42bb-9365-b3f9006fe560 admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:07.783872 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e99a8-7ee0-42bb-9365-b3f9006fe560 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:30:07.784179 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e99a8-7ee0-42bb-9365-b3f9006fe560 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:07.785839 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e99a8-7ee0-42bb-9365-b3f9006fe560 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:07.817132 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1002/1999] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:30:07.822454 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c1b67-0813-4042-ad07-00d884d30a53 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:07.836043 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c1b67-0813-4042-ad07-00d884d30a53 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:07.843540 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a509f57c-15c4-4631-bfa1-ad3fbfe571c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:07.844765 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509f57c-15c4-4631-bfa1-ad3fbfe571c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:07.844999 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509f57c-15c4-4631-bfa1-ad3fbfe571c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:07.845416 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509f57c-15c4-4631-bfa1-ad3fbfe571c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:07.848089 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a509f57c-15c4-4631-bfa1-ad3fbfe571c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:07.850020 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a509f57c-15c4-4631-bfa1-ad3fbfe571c4 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:07.886456 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 998/2000] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:07 2022] POST /identity/v3/projects => generated 399 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:07.890742 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e99a8-7ee0-42bb-9365-b3f9006fe560 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:07.903601 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e99a8-7ee0-42bb-9365-b3f9006fe560 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:07.929222 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:07.987385 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6ed2b-3d2e-4af7-9770-f1ff8e199c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:07.987751 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6ed2b-3d2e-4af7-9770-f1ff8e199c19 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:07.988030 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6ed2b-3d2e-4af7-9770-f1ff8e199c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:07.988302 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6ed2b-3d2e-4af7-9770-f1ff8e199c19 admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:07.988945 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6ed2b-3d2e-4af7-9770-f1ff8e199c19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:07.991014 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6ed2b-3d2e-4af7-9770-f1ff8e199c19 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:08.020953 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1003/2001] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:07 2022] POST /identity/v3/users => generated 300 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:08.022187 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a509f57c-15c4-4631-bfa1-ad3fbfe571c4 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:08.033029 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a509f57c-15c4-4631-bfa1-ad3fbfe571c4 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:08.038483 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ac458-d731-4490-941c-32f259f56279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:08.040721 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ac458-d731-4490-941c-32f259f56279 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:08.040996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ac458-d731-4490-941c-32f259f56279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:08.041269 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ac458-d731-4490-941c-32f259f56279 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:08.041958 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39ac458-d731-4490-941c-32f259f56279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:08.043566 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39ac458-d731-4490-941c-32f259f56279 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:08.050324 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:08.056203 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 999/2002] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:08.060429 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6ed2b-3d2e-4af7-9770-f1ff8e199c19 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:08.084031 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6ed2b-3d2e-4af7-9770-f1ff8e199c19 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:08.089817 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c0e7f7-3250-47a7-ba7e-1b4e315c1a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:08.090792 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0e7f7-3250-47a7-ba7e-1b4e315c1a7c admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:08.090792 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0e7f7-3250-47a7-ba7e-1b4e315c1a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:08.090792 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0e7f7-3250-47a7-ba7e-1b4e315c1a7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:08.091878 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c0e7f7-3250-47a7-ba7e-1b4e315c1a7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:08.093254 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c0e7f7-3250-47a7-ba7e-1b4e315c1a7c admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:08.107878 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1004/2003] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:08.109050 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ac458-d731-4490-941c-32f259f56279 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:08.119957 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ac458-d731-4490-941c-32f259f56279 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:08.124532 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb62e4e9-ee1e-4c59-8134-01082c55b2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:08.125144 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62e4e9-ee1e-4c59-8134-01082c55b2cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:08.125422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62e4e9-ee1e-4c59-8134-01082c55b2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:08.125857 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62e4e9-ee1e-4c59-8134-01082c55b2cd admin admin] PATH_INFO: `/v3/projects/11267524c7534eab8ede077f99487031/users/9b0954c4f291402b9c5326e8a481973d/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:08.153496 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb62e4e9-ee1e-4c59-8134-01082c55b2cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=11267524c7534eab8ede077f99487031, user_id=9b0954c4f291402b9c5326e8a481973d, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:08.159561 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb62e4e9-ee1e-4c59-8134-01082c55b2cd admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:08.180360 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1000/2004] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:08 2022] PUT /identity/v3/projects/11267524c7534eab8ede077f99487031/users/9b0954c4f291402b9c5326e8a481973d/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:08.191545 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243e76e-92b0-47fe-91d9-2fb30f1ca03e None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:08.191984 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243e76e-92b0-47fe-91d9-2fb30f1ca03e None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:08.192440 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243e76e-92b0-47fe-91d9-2fb30f1ca03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:08.224970 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-0243e76e-92b0-47fe-91d9-2fb30f1ca03e None None] MFA Rules not processed for user `9b0954c4f291402b9c5326e8a481973d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:08.226281 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-0243e76e-92b0-47fe-91d9-2fb30f1ca03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:08.324692 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1005/2005] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:08 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:08.351736 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb62e4e9-ee1e-4c59-8134-01082c55b2cd admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:08.353592 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-fb62e4e9-ee1e-4c59-8134-01082c55b2cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:08.417248 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb62e4e9-ee1e-4c59-8134-01082c55b2cd admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:08.422486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb3bea-4d97-4a2a-b36a-ae024af25a3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:08.422486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb3bea-4d97-4a2a-b36a-ae024af25a3d service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:08.422770 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb3bea-4d97-4a2a-b36a-ae024af25a3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:08.422866 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb3bea-4d97-4a2a-b36a-ae024af25a3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:08.433639 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eb3bea-4d97-4a2a-b36a-ae024af25a3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:08.435860 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eb3bea-4d97-4a2a-b36a-ae024af25a3d service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:08.468591 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1001/2006] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:08 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:10.853420 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0243e76e-92b0-47fe-91d9-2fb30f1ca03e None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:10.865943 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-0243e76e-92b0-47fe-91d9-2fb30f1ca03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:11.083417 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0243e76e-92b0-47fe-91d9-2fb30f1ca03e None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:11.083417 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7d3eb-5b13-4246-acb7-6d10cde44751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:11.083417 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7d3eb-5b13-4246-acb7-6d10cde44751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:11.083417 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7d3eb-5b13-4246-acb7-6d10cde44751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:11.084075 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7d3eb-5b13-4246-acb7-6d10cde44751 admin admin] PATH_INFO: `/v3/users/0a6c7f817c564e9ebc6e313d43d5325c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:11.085133 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f40876-675b-4b1c-a1f8-f2042f71dfd5 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:11.085624 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f40876-675b-4b1c-a1f8-f2042f71dfd5 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:11.085728 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f40876-675b-4b1c-a1f8-f2042f71dfd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:11.088926 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a7d3eb-5b13-4246-acb7-6d10cde44751 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a6c7f817c564e9ebc6e313d43d5325c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:11.091105 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a7d3eb-5b13-4246-acb7-6d10cde44751 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:11.176065 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-08f40876-675b-4b1c-a1f8-f2042f71dfd5 None None] MFA Rules not processed for user `d101c78015b842f99ea0662f60aa7283`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:11.177586 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-08f40876-675b-4b1c-a1f8-f2042f71dfd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:11.183271 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:11.211775 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-07a7d3eb-5b13-4246-acb7-6d10cde44751 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a6c7f817c564e9ebc6e313d43d5325c'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:11.223785 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-07a7d3eb-5b13-4246-acb7-6d10cde44751 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a6c7f817c564e9ebc6e313d43d5325c'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:11.224537 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-07a7d3eb-5b13-4246-acb7-6d10cde44751 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a6c7f817c564e9ebc6e313d43d5325c'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:11.256038 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-07a7d3eb-5b13-4246-acb7-6d10cde44751 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a6c7f817c564e9ebc6e313d43d5325c'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:11.290012 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-07a7d3eb-5b13-4246-acb7-6d10cde44751 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a6c7f817c564e9ebc6e313d43d5325c'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:11.304747 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1006/2007] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:10 2022] DELETE /identity/v3/users/0a6c7f817c564e9ebc6e313d43d5325c => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:11.306486 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e0148-104b-4e80-9a7f-a26a797b0849 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:11.307690 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e0148-104b-4e80-9a7f-a26a797b0849 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:11.307821 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e0148-104b-4e80-9a7f-a26a797b0849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:11.356657 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-2e0e0148-104b-4e80-9a7f-a26a797b0849 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:11.357861 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-2e0e0148-104b-4e80-9a7f-a26a797b0849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:11.443945 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1002/2008] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:11 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:11.458357 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f40876-675b-4b1c-a1f8-f2042f71dfd5 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:11.459967 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-08f40876-675b-4b1c-a1f8-f2042f71dfd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:11.624751 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1007/2009] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:30:11.638274 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e0148-104b-4e80-9a7f-a26a797b0849 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:11.656993 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-2e0e0148-104b-4e80-9a7f-a26a797b0849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:11.809392 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:11.833595 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f40876-675b-4b1c-a1f8-f2042f71dfd5 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:11.881282 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e0148-104b-4e80-9a7f-a26a797b0849 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:11.898029 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028e2d3-f97f-42ed-9d2c-dd6cbb3ac592 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:11.901289 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028e2d3-f97f-42ed-9d2c-dd6cbb3ac592 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:11.904800 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028e2d3-f97f-42ed-9d2c-dd6cbb3ac592 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:11.905294 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028e2d3-f97f-42ed-9d2c-dd6cbb3ac592 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:11.953373 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1028e2d3-f97f-42ed-9d2c-dd6cbb3ac592 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:11.957295 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1028e2d3-f97f-42ed-9d2c-dd6cbb3ac592 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:11.985606 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1008/2010] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:12.055435 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b465c53-f579-4896-8c71-b6ca690c0d33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:12.056485 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b465c53-f579-4896-8c71-b6ca690c0d33 service nova] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:12.058219 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b465c53-f579-4896-8c71-b6ca690c0d33 service nova] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:12.060753 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b465c53-f579-4896-8c71-b6ca690c0d33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:12.068707 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-9b465c53-f579-4896-8c71-b6ca690c0d33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:12.280039 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b465c53-f579-4896-8c71-b6ca690c0d33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:12.287334 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b465c53-f579-4896-8c71-b6ca690c0d33 service nova] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:12.319928 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1003/2011] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 862 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:30:13.206424 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028e2d3-f97f-42ed-9d2c-dd6cbb3ac592 service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:13.209246 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-1028e2d3-f97f-42ed-9d2c-dd6cbb3ac592 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:13.227314 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028e2d3-f97f-42ed-9d2c-dd6cbb3ac592 service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:13.233884 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e131ae93-3e58-4d97-af08-e8a1a0060984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:13.235064 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131ae93-3e58-4d97-af08-e8a1a0060984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:13.235527 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131ae93-3e58-4d97-af08-e8a1a0060984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:13.235962 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131ae93-3e58-4d97-af08-e8a1a0060984 admin admin] PATH_INFO: `/v3/projects/7f5f2149dcc54fa696e2dfc60f0c6dd0` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:13.237216 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e131ae93-3e58-4d97-af08-e8a1a0060984 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f5f2149dcc54fa696e2dfc60f0c6dd0)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:13.239290 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e131ae93-3e58-4d97-af08-e8a1a0060984 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:13.354368 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-e131ae93-3e58-4d97-af08-e8a1a0060984 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f5f2149dcc54fa696e2dfc60f0c6dd0', 'request_id': 'req-e131ae93-3e58-4d97-af08-e8a1a0060984'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:13.355914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:13.363875 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-e131ae93-3e58-4d97-af08-e8a1a0060984 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f5f2149dcc54fa696e2dfc60f0c6dd0', 'request_id': 'req-e131ae93-3e58-4d97-af08-e8a1a0060984'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:13.372574 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-e131ae93-3e58-4d97-af08-e8a1a0060984 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f5f2149dcc54fa696e2dfc60f0c6dd0', 'request_id': 'req-e131ae93-3e58-4d97-af08-e8a1a0060984'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:13.387386 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-e131ae93-3e58-4d97-af08-e8a1a0060984 admin admin] The token cache is being invalidate because project 7f5f2149dcc54fa696e2dfc60f0c6dd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:13.387681 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-e131ae93-3e58-4d97-af08-e8a1a0060984 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:13.388856 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-e131ae93-3e58-4d97-af08-e8a1a0060984 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:13.390395 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1009/2012] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:13 2022] DELETE /identity/v3/projects/7f5f2149dcc54fa696e2dfc60f0c6dd0 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:13.395339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711df7b9-b308-49da-9694-81d348c8f056 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:13.395883 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711df7b9-b308-49da-9694-81d348c8f056 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:13.395883 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711df7b9-b308-49da-9694-81d348c8f056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:13.423396 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-711df7b9-b308-49da-9694-81d348c8f056 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:13.426852 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-711df7b9-b308-49da-9694-81d348c8f056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:13.532719 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1004/2013] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:13.540479 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba5151-c5e3-4fdf-b8a1-c7b7daf9994c None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:13.540763 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba5151-c5e3-4fdf-b8a1-c7b7daf9994c None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:13.541039 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba5151-c5e3-4fdf-b8a1-c7b7daf9994c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:13.569213 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-72ba5151-c5e3-4fdf-b8a1-c7b7daf9994c None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:13.570444 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-72ba5151-c5e3-4fdf-b8a1-c7b7daf9994c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:13.585814 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1010/2014] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:30:13.586996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711df7b9-b308-49da-9694-81d348c8f056 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:13.611745 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711df7b9-b308-49da-9694-81d348c8f056 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:13.616647 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13e5c2-84b8-461e-9d9d-1295140a6281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:13.617512 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13e5c2-84b8-461e-9d9d-1295140a6281 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:13.618006 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13e5c2-84b8-461e-9d9d-1295140a6281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:13.619011 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13e5c2-84b8-461e-9d9d-1295140a6281 admin admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:13.619984 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c13e5c2-84b8-461e-9d9d-1295140a6281 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:30:13.620443 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c13e5c2-84b8-461e-9d9d-1295140a6281 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:13.622443 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c13e5c2-84b8-461e-9d9d-1295140a6281 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:13.651462 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1005/2015] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:30:13.654553 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba5151-c5e3-4fdf-b8a1-c7b7daf9994c None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:13.666482 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba5151-c5e3-4fdf-b8a1-c7b7daf9994c None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:13.678512 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5714f-7ad5-42d6-8262-efd7d82894ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:13.679852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5714f-7ad5-42d6-8262-efd7d82894ca admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:13.680445 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5714f-7ad5-42d6-8262-efd7d82894ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:13.681249 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5714f-7ad5-42d6-8262-efd7d82894ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:13.682331 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f5714f-7ad5-42d6-8262-efd7d82894ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:13.685560 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f5714f-7ad5-42d6-8262-efd7d82894ca admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:13.767425 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13e5c2-84b8-461e-9d9d-1295140a6281 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:13.778784 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1011/2016] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:13 2022] POST /identity/v3/projects => generated 369 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:13.781302 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13e5c2-84b8-461e-9d9d-1295140a6281 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:13.815861 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:13.880592 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dbb97f-0be1-4d0a-bf51-201f295149a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:13.881234 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dbb97f-0be1-4d0a-bf51-201f295149a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:13.881683 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dbb97f-0be1-4d0a-bf51-201f295149a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:13.881955 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dbb97f-0be1-4d0a-bf51-201f295149a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:13.882601 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dbb97f-0be1-4d0a-bf51-201f295149a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:13.885190 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dbb97f-0be1-4d0a-bf51-201f295149a1 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:13.915439 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1006/2017] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:13 2022] POST /identity/v3/users => generated 285 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:13.919440 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5714f-7ad5-42d6-8262-efd7d82894ca admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:13.933216 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5714f-7ad5-42d6-8262-efd7d82894ca admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:13.940235 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc6ea73-cd2b-48b7-81c1-2b34dc83406d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:13.940873 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6ea73-cd2b-48b7-81c1-2b34dc83406d admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:13.941245 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6ea73-cd2b-48b7-81c1-2b34dc83406d admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:13.941519 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6ea73-cd2b-48b7-81c1-2b34dc83406d admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:13.942168 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc6ea73-cd2b-48b7-81c1-2b34dc83406d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:13.944644 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc6ea73-cd2b-48b7-81c1-2b34dc83406d admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:13.959447 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1012/2018] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:13.963509 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dbb97f-0be1-4d0a-bf51-201f295149a1 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:13.975150 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dbb97f-0be1-4d0a-bf51-201f295149a1 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:13.981055 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdada44-0437-4083-9a03-0b7d93b65c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:13.982947 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdada44-0437-4083-9a03-0b7d93b65c33 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:13.983362 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdada44-0437-4083-9a03-0b7d93b65c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:13.983702 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdada44-0437-4083-9a03-0b7d93b65c33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:13.984556 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdada44-0437-4083-9a03-0b7d93b65c33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:13.986336 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdada44-0437-4083-9a03-0b7d93b65c33 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:13.997926 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1007/2019] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:14.002713 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc6ea73-cd2b-48b7-81c1-2b34dc83406d admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:14.012955 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc6ea73-cd2b-48b7-81c1-2b34dc83406d admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:14.018382 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5c9431-0c86-4593-aad9-0543863085ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:14.019102 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c9431-0c86-4593-aad9-0543863085ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:14.019392 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c9431-0c86-4593-aad9-0543863085ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:14.019777 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c9431-0c86-4593-aad9-0543863085ce admin admin] PATH_INFO: `/v3/projects/e9426908dddc440cb2092287386250e9/users/fbfdde83696d486eb39384d1bb01bf8e/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:14.036130 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5c9431-0c86-4593-aad9-0543863085ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9426908dddc440cb2092287386250e9, user_id=fbfdde83696d486eb39384d1bb01bf8e, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:14.039119 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5c9431-0c86-4593-aad9-0543863085ce admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:14.055520 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1013/2020] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:14 2022] PUT /identity/v3/projects/e9426908dddc440cb2092287386250e9/users/fbfdde83696d486eb39384d1bb01bf8e/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:14.128139 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d92b37-9c9e-439a-9bf5-38901ea5dd8d None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:14.128352 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d92b37-9c9e-439a-9bf5-38901ea5dd8d None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:14.128635 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d92b37-9c9e-439a-9bf5-38901ea5dd8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:14.162525 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-e9d92b37-9c9e-439a-9bf5-38901ea5dd8d None None] MFA Rules not processed for user `fbfdde83696d486eb39384d1bb01bf8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:14.163952 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-e9d92b37-9c9e-439a-9bf5-38901ea5dd8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:14.256775 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1008/2021] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:14.265237 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5c9431-0c86-4593-aad9-0543863085ce admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:14.267069 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-0e5c9431-0c86-4593-aad9-0543863085ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:14.342836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5c9431-0c86-4593-aad9-0543863085ce admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:14.347251 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2999c8c6-06e6-476d-94a7-482cccf33023 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a051ca8154540ad93db6d028b8323a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:14.348016 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2999c8c6-06e6-476d-94a7-482cccf33023 service glance] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:14.348461 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2999c8c6-06e6-476d-94a7-482cccf33023 service glance] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:14.348791 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2999c8c6-06e6-476d-94a7-482cccf33023 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:14.359771 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2999c8c6-06e6-476d-94a7-482cccf33023 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:14.362080 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2999c8c6-06e6-476d-94a7-482cccf33023 service glance] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:14.371918 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1014/2022] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:14 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:17.863582 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d92b37-9c9e-439a-9bf5-38901ea5dd8d None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:17.865052 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-e9d92b37-9c9e-439a-9bf5-38901ea5dd8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:17.960867 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d92b37-9c9e-439a-9bf5-38901ea5dd8d None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:17.968630 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9762f782-25d5-446b-8d2a-01c7eee588e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:17.970730 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762f782-25d5-446b-8d2a-01c7eee588e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:17.970730 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762f782-25d5-446b-8d2a-01c7eee588e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:17.970730 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762f782-25d5-446b-8d2a-01c7eee588e0 admin admin] PATH_INFO: `/v3/users/30d8e7e1500f4e98935f3dd366488b0b` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:17.971169 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9762f782-25d5-446b-8d2a-01c7eee588e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30d8e7e1500f4e98935f3dd366488b0b)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:17.973179 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9762f782-25d5-446b-8d2a-01c7eee588e0 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:18.020581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:18.042186 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-9762f782-25d5-446b-8d2a-01c7eee588e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30d8e7e1500f4e98935f3dd366488b0b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:18.043035 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-9762f782-25d5-446b-8d2a-01c7eee588e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30d8e7e1500f4e98935f3dd366488b0b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:18.051532 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-9762f782-25d5-446b-8d2a-01c7eee588e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30d8e7e1500f4e98935f3dd366488b0b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:18.066594 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-9762f782-25d5-446b-8d2a-01c7eee588e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30d8e7e1500f4e98935f3dd366488b0b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:18.077346 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-9762f782-25d5-446b-8d2a-01c7eee588e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30d8e7e1500f4e98935f3dd366488b0b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:18.084864 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1009/2023] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:17 2022] DELETE /identity/v3/users/30d8e7e1500f4e98935f3dd366488b0b => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:18.089323 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44135a9d-4bd9-46f0-bfe6-68578c02869b None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:18.089829 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44135a9d-4bd9-46f0-bfe6-68578c02869b None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:18.089970 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44135a9d-4bd9-46f0-bfe6-68578c02869b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:18.130625 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-44135a9d-4bd9-46f0-bfe6-68578c02869b None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:18.131863 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-44135a9d-4bd9-46f0-bfe6-68578c02869b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:18.200636 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1015/2024] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:18.211371 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9762f782-25d5-446b-8d2a-01c7eee588e0 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:18.212765 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-9762f782-25d5-446b-8d2a-01c7eee588e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:18.282740 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9762f782-25d5-446b-8d2a-01c7eee588e0 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:18.286470 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad9f8f-719b-485d-9e3a-3a7f9d00e7c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:18.287387 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad9f8f-719b-485d-9e3a-3a7f9d00e7c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:18.287662 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad9f8f-719b-485d-9e3a-3a7f9d00e7c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:18.287934 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad9f8f-719b-485d-9e3a-3a7f9d00e7c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:18.299089 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad9f8f-719b-485d-9e3a-3a7f9d00e7c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:18.300864 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad9f8f-719b-485d-9e3a-3a7f9d00e7c1 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:18.311383 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1010/2025] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:18.603150 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44135a9d-4bd9-46f0-bfe6-68578c02869b None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:18.607847 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-44135a9d-4bd9-46f0-bfe6-68578c02869b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:18.609043 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad9f8f-719b-485d-9e3a-3a7f9d00e7c1 service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:18.610794 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-8dad9f8f-719b-485d-9e3a-3a7f9d00e7c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:18.629243 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44135a9d-4bd9-46f0-bfe6-68578c02869b None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:18.634388 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a30441-4901-4de1-b867-5d7fedc20706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:18.635062 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a30441-4901-4de1-b867-5d7fedc20706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:18.635333 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a30441-4901-4de1-b867-5d7fedc20706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:18.635596 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a30441-4901-4de1-b867-5d7fedc20706 admin admin] PATH_INFO: `/v3/users/23adbcf72828456daadee6a5dd1e5545` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:18.636696 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a30441-4901-4de1-b867-5d7fedc20706 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23adbcf72828456daadee6a5dd1e5545)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:18.637406 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad9f8f-719b-485d-9e3a-3a7f9d00e7c1 service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:18.638627 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a30441-4901-4de1-b867-5d7fedc20706 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:18.643339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69894430-7d0b-4df7-ae89-293f78d3556c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:18.644064 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69894430-7d0b-4df7-ae89-293f78d3556c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:18.644403 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69894430-7d0b-4df7-ae89-293f78d3556c admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:18.644757 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69894430-7d0b-4df7-ae89-293f78d3556c admin admin] PATH_INFO: `/v3/projects/8fe580d37fe7404b8f386f66e0e96c72` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:18.646096 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69894430-7d0b-4df7-ae89-293f78d3556c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fe580d37fe7404b8f386f66e0e96c72)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:18.648194 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69894430-7d0b-4df7-ae89-293f78d3556c admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:18.654871 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:INSERT=3,keystone:DELETE=4,keystone:SELECT=93 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:18.700640 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-30a30441-4901-4de1-b867-5d7fedc20706 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23adbcf72828456daadee6a5dd1e5545'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:18.711877 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-30a30441-4901-4de1-b867-5d7fedc20706 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23adbcf72828456daadee6a5dd1e5545'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:18.712806 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-30a30441-4901-4de1-b867-5d7fedc20706 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23adbcf72828456daadee6a5dd1e5545'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:18.715205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-69894430-7d0b-4df7-ae89-293f78d3556c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fe580d37fe7404b8f386f66e0e96c72', 'request_id': 'req-69894430-7d0b-4df7-ae89-293f78d3556c'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:18.722319 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-69894430-7d0b-4df7-ae89-293f78d3556c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fe580d37fe7404b8f386f66e0e96c72', 'request_id': 'req-69894430-7d0b-4df7-ae89-293f78d3556c'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:18.731792 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-69894430-7d0b-4df7-ae89-293f78d3556c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fe580d37fe7404b8f386f66e0e96c72', 'request_id': 'req-69894430-7d0b-4df7-ae89-293f78d3556c'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:18.733740 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-30a30441-4901-4de1-b867-5d7fedc20706 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23adbcf72828456daadee6a5dd1e5545'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:18.749512 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-69894430-7d0b-4df7-ae89-293f78d3556c admin admin] The token cache is being invalidate because project 8fe580d37fe7404b8f386f66e0e96c72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:18.749921 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-69894430-7d0b-4df7-ae89-293f78d3556c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:18.753998 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-30a30441-4901-4de1-b867-5d7fedc20706 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23adbcf72828456daadee6a5dd1e5545'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:18.761262 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-69894430-7d0b-4df7-ae89-293f78d3556c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:18.777366 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1011/2026] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:18 2022] DELETE /identity/v3/projects/8fe580d37fe7404b8f386f66e0e96c72 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:30:18.777366 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69894430-7d0b-4df7-ae89-293f78d3556c admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:18.777366 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-69894430-7d0b-4df7-ae89-293f78d3556c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:18.785730 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1016/2027] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:18 2022] DELETE /identity/v3/users/23adbcf72828456daadee6a5dd1e5545 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:18.788124 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c0a54-8d25-4be0-82a1-3276924e1113 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:18.788630 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c0a54-8d25-4be0-82a1-3276924e1113 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:18.791547 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c0a54-8d25-4be0-82a1-3276924e1113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:18.816908 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-b32c0a54-8d25-4be0-82a1-3276924e1113 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:18.818536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-b32c0a54-8d25-4be0-82a1-3276924e1113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:18.862788 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:18.901366 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69894430-7d0b-4df7-ae89-293f78d3556c admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:18.934729 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1017/2028] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:18.944788 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c0a54-8d25-4be0-82a1-3276924e1113 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:18.946472 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-b32c0a54-8d25-4be0-82a1-3276924e1113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:19.011844 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa3dcc-d1a5-4079-8d10-7201ddb22ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:19.012795 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa3dcc-d1a5-4079-8d10-7201ddb22ba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:19.013277 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa3dcc-d1a5-4079-8d10-7201ddb22ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:19.013604 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa3dcc-d1a5-4079-8d10-7201ddb22ba1 admin admin] PATH_INFO: `/v3/users/c06ec777b52448d3bf4f1952a7b7b65b` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:19.014996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa3dcc-d1a5-4079-8d10-7201ddb22ba1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c06ec777b52448d3bf4f1952a7b7b65b)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:19.017582 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa3dcc-d1a5-4079-8d10-7201ddb22ba1 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:19.028084 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c0a54-8d25-4be0-82a1-3276924e1113 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:19.032487 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d34d57-92cb-473a-ba6b-5851da8061f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:19.033368 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d34d57-92cb-473a-ba6b-5851da8061f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:19.033530 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d34d57-92cb-473a-ba6b-5851da8061f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:19.033683 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d34d57-92cb-473a-ba6b-5851da8061f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:19.048721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d34d57-92cb-473a-ba6b-5851da8061f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:19.051028 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d34d57-92cb-473a-ba6b-5851da8061f1 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:19.066591 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1018/2029] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:19.067846 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d34d57-92cb-473a-ba6b-5851da8061f1 service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:19.069209 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-c9d34d57-92cb-473a-ba6b-5851da8061f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:19.083398 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d34d57-92cb-473a-ba6b-5851da8061f1 service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:19.086920 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-feaa3dcc-d1a5-4079-8d10-7201ddb22ba1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c06ec777b52448d3bf4f1952a7b7b65b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:19.088929 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666207e2-434f-401f-8c86-f42cc01eca81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:19.089586 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666207e2-434f-401f-8c86-f42cc01eca81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:19.090220 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666207e2-434f-401f-8c86-f42cc01eca81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:19.090220 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666207e2-434f-401f-8c86-f42cc01eca81 admin admin] PATH_INFO: `/v3/users/9b0954c4f291402b9c5326e8a481973d` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:19.091414 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666207e2-434f-401f-8c86-f42cc01eca81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b0954c4f291402b9c5326e8a481973d)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:19.093295 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-feaa3dcc-d1a5-4079-8d10-7201ddb22ba1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c06ec777b52448d3bf4f1952a7b7b65b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:19.093473 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666207e2-434f-401f-8c86-f42cc01eca81 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:19.103345 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-feaa3dcc-d1a5-4079-8d10-7201ddb22ba1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c06ec777b52448d3bf4f1952a7b7b65b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:19.119412 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-feaa3dcc-d1a5-4079-8d10-7201ddb22ba1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c06ec777b52448d3bf4f1952a7b7b65b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:19.134606 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:19.142604 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-feaa3dcc-d1a5-4079-8d10-7201ddb22ba1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c06ec777b52448d3bf4f1952a7b7b65b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:19.156369 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1012/2030] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:18 2022] DELETE /identity/v3/users/c06ec777b52448d3bf4f1952a7b7b65b => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:19.187807 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-666207e2-434f-401f-8c86-f42cc01eca81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b0954c4f291402b9c5326e8a481973d'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:19.197599 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-666207e2-434f-401f-8c86-f42cc01eca81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b0954c4f291402b9c5326e8a481973d'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:19.198433 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-666207e2-434f-401f-8c86-f42cc01eca81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b0954c4f291402b9c5326e8a481973d'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:19.215877 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-666207e2-434f-401f-8c86-f42cc01eca81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b0954c4f291402b9c5326e8a481973d'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:19.229678 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-666207e2-434f-401f-8c86-f42cc01eca81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b0954c4f291402b9c5326e8a481973d'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:19.238459 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1019/2031] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:19 2022] DELETE /identity/v3/users/9b0954c4f291402b9c5326e8a481973d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:19.243230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81b4a5-6555-4c54-8758-5990eb128378 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:19.243515 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81b4a5-6555-4c54-8758-5990eb128378 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:19.243784 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81b4a5-6555-4c54-8758-5990eb128378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:19.270684 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-fc81b4a5-6555-4c54-8758-5990eb128378 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:19.270684 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-fc81b4a5-6555-4c54-8758-5990eb128378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:19.362838 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1013/2032] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:30:19.370619 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666207e2-434f-401f-8c86-f42cc01eca81 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:19.386960 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-666207e2-434f-401f-8c86-f42cc01eca81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:19.427332 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81b4a5-6555-4c54-8758-5990eb128378 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:19.429044 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-fc81b4a5-6555-4c54-8758-5990eb128378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:19.454819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81b4a5-6555-4c54-8758-5990eb128378 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:19.461227 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849a4c0-db88-4ea1-8de0-335d83de0484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:19.462153 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849a4c0-db88-4ea1-8de0-335d83de0484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:19.462592 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849a4c0-db88-4ea1-8de0-335d83de0484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:19.463344 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849a4c0-db88-4ea1-8de0-335d83de0484 admin admin] PATH_INFO: `/v3/projects/100f071407274b6aaeaaa8ed758df75f` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:19.464804 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0849a4c0-db88-4ea1-8de0-335d83de0484 admin admin] RBAC: Authorizing `identity:delete_project(project_id=100f071407274b6aaeaaa8ed758df75f)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:19.467078 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0849a4c0-db88-4ea1-8de0-335d83de0484 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:19.469659 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:19.487830 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666207e2-434f-401f-8c86-f42cc01eca81 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:19.498749 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51367e7-78c1-4abc-93c0-8a2183c52e4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:19.499514 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51367e7-78c1-4abc-93c0-8a2183c52e4b service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:19.499793 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51367e7-78c1-4abc-93c0-8a2183c52e4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:19.500065 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51367e7-78c1-4abc-93c0-8a2183c52e4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:19.511764 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51367e7-78c1-4abc-93c0-8a2183c52e4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:19.530317 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51367e7-78c1-4abc-93c0-8a2183c52e4b service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:19.537982 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-0849a4c0-db88-4ea1-8de0-335d83de0484 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '100f071407274b6aaeaaa8ed758df75f', 'request_id': 'req-0849a4c0-db88-4ea1-8de0-335d83de0484'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:19.550307 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-0849a4c0-db88-4ea1-8de0-335d83de0484 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '100f071407274b6aaeaaa8ed758df75f', 'request_id': 'req-0849a4c0-db88-4ea1-8de0-335d83de0484'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:19.560449 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-0849a4c0-db88-4ea1-8de0-335d83de0484 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '100f071407274b6aaeaaa8ed758df75f', 'request_id': 'req-0849a4c0-db88-4ea1-8de0-335d83de0484'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:19.564698 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1020/2033] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:19.582227 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-0849a4c0-db88-4ea1-8de0-335d83de0484 admin admin] The token cache is being invalidate because project 100f071407274b6aaeaaa8ed758df75f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:19.582538 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-0849a4c0-db88-4ea1-8de0-335d83de0484 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:19.583260 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-0849a4c0-db88-4ea1-8de0-335d83de0484 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:19.589801 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1014/2034] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:19 2022] DELETE /identity/v3/projects/100f071407274b6aaeaaa8ed758df75f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:19.598163 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf927830-4da7-472d-bbcb-11b472780b6d None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:19.598765 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf927830-4da7-472d-bbcb-11b472780b6d None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:19.599267 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf927830-4da7-472d-bbcb-11b472780b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:19.653543 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-bf927830-4da7-472d-bbcb-11b472780b6d None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:19.655189 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-bf927830-4da7-472d-bbcb-11b472780b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:19.753054 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1021/2035] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:19.761286 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e2099-c45a-4d0a-afc0-87f7575a1b0a None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:19.761672 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e2099-c45a-4d0a-afc0-87f7575a1b0a None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:19.761992 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e2099-c45a-4d0a-afc0-87f7575a1b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:19.783606 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-f31e2099-c45a-4d0a-afc0-87f7575a1b0a None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:19.784853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-f31e2099-c45a-4d0a-afc0-87f7575a1b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:19.793807 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1015/2036] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:19.798812 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf927830-4da7-472d-bbcb-11b472780b6d None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:19.808245 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf927830-4da7-472d-bbcb-11b472780b6d None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:19.813515 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77511b-8282-4c12-ad08-0d6e76594c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:19.814190 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77511b-8282-4c12-ad08-0d6e76594c7f admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:19.814461 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77511b-8282-4c12-ad08-0d6e76594c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:19.815744 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77511b-8282-4c12-ad08-0d6e76594c7f admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:19.824753 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b77511b-8282-4c12-ad08-0d6e76594c7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:30:19.825089 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b77511b-8282-4c12-ad08-0d6e76594c7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:19.828108 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b77511b-8282-4c12-ad08-0d6e76594c7f admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:19.843078 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1022/2037] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:30:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:30:19.849125 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e2099-c45a-4d0a-afc0-87f7575a1b0a None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:19.855429 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77511b-8282-4c12-ad08-0d6e76594c7f admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:19.857500 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-6b77511b-8282-4c12-ad08-0d6e76594c7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:19.868669 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e2099-c45a-4d0a-afc0-87f7575a1b0a None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:19.871577 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77511b-8282-4c12-ad08-0d6e76594c7f admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:19.873943 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1cd5a-3c02-4ae3-86c5-d98b08da71d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:19.874623 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1cd5a-3c02-4ae3-86c5-d98b08da71d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:19.875502 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1cd5a-3c02-4ae3-86c5-d98b08da71d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:19.876478 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1cd5a-3c02-4ae3-86c5-d98b08da71d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:19.877145 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d1cd5a-3c02-4ae3-86c5-d98b08da71d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:19.877367 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1faf7c-4245-4071-abe6-93671a698be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:19.878130 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1faf7c-4245-4071-abe6-93671a698be6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:19.878466 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1faf7c-4245-4071-abe6-93671a698be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:19.878836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1faf7c-4245-4071-abe6-93671a698be6 admin admin] PATH_INFO: `/v3/projects/b35c6964108140d69f68ed4d9d7a37d0` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:19.879465 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d1cd5a-3c02-4ae3-86c5-d98b08da71d3 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:19.883692 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1faf7c-4245-4071-abe6-93671a698be6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b35c6964108140d69f68ed4d9d7a37d0)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:19.886240 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1faf7c-4245-4071-abe6-93671a698be6 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:19.921069 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1016/2038] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:19 2022] POST /identity/v3/projects => generated 383 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:19.926464 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1cd5a-3c02-4ae3-86c5-d98b08da71d3 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:19.931382 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:19.965592 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-6d1faf7c-4245-4071-abe6-93671a698be6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b35c6964108140d69f68ed4d9d7a37d0', 'request_id': 'req-6d1faf7c-4245-4071-abe6-93671a698be6'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:19.980867 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-6d1faf7c-4245-4071-abe6-93671a698be6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b35c6964108140d69f68ed4d9d7a37d0', 'request_id': 'req-6d1faf7c-4245-4071-abe6-93671a698be6'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:20.000170 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-6d1faf7c-4245-4071-abe6-93671a698be6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b35c6964108140d69f68ed4d9d7a37d0', 'request_id': 'req-6d1faf7c-4245-4071-abe6-93671a698be6'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:20.027117 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-6d1faf7c-4245-4071-abe6-93671a698be6 admin admin] The token cache is being invalidate because project b35c6964108140d69f68ed4d9d7a37d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:20.027422 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-6d1faf7c-4245-4071-abe6-93671a698be6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:20.028167 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-6d1faf7c-4245-4071-abe6-93671a698be6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:20.030062 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1023/2039] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:19 2022] DELETE /identity/v3/projects/b35c6964108140d69f68ed4d9d7a37d0 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:20.036282 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ccfd6-74b5-44d3-b869-8358afe83d49 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:20.036856 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ccfd6-74b5-44d3-b869-8358afe83d49 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:20.037258 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ccfd6-74b5-44d3-b869-8358afe83d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:20.050089 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1cd5a-3c02-4ae3-86c5-d98b08da71d3 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:20.064271 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-8b6ccfd6-74b5-44d3-b869-8358afe83d49 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:20.065793 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-8b6ccfd6-74b5-44d3-b869-8358afe83d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:20.174520 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1024/2040] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:30:20.180712 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ccfd6-74b5-44d3-b869-8358afe83d49 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:20.182298 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-8b6ccfd6-74b5-44d3-b869-8358afe83d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:20.196116 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c1a26-4710-4a13-a661-d6fa9510b5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:20.197204 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c1a26-4710-4a13-a661-d6fa9510b5bf admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:20.197706 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c1a26-4710-4a13-a661-d6fa9510b5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:20.198105 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c1a26-4710-4a13-a661-d6fa9510b5bf admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:20.198804 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c1a26-4710-4a13-a661-d6fa9510b5bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:20.199873 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ccfd6-74b5-44d3-b869-8358afe83d49 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:20.200766 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c1a26-4710-4a13-a661-d6fa9510b5bf admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:20.208442 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ed5bde-9982-4357-b737-50b832738db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:20.208951 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed5bde-9982-4357-b737-50b832738db8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:20.209319 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed5bde-9982-4357-b737-50b832738db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:20.209704 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed5bde-9982-4357-b737-50b832738db8 admin admin] PATH_INFO: `/v3/projects/11267524c7534eab8ede077f99487031` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:20.211513 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ed5bde-9982-4357-b737-50b832738db8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11267524c7534eab8ede077f99487031)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:20.213532 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ed5bde-9982-4357-b737-50b832738db8 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:20.221632 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:20.234412 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1017/2041] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:19 2022] POST /identity/v3/users => generated 292 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:20.236058 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f39155-b076-4b8f-aaad-4b7b21666059 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:20.236481 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f39155-b076-4b8f-aaad-4b7b21666059 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:20.237117 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f39155-b076-4b8f-aaad-4b7b21666059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:20.277895 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-e9f39155-b076-4b8f-aaad-4b7b21666059 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:20.279340 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-e9f39155-b076-4b8f-aaad-4b7b21666059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:20.286926 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-a2ed5bde-9982-4357-b737-50b832738db8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11267524c7534eab8ede077f99487031', 'request_id': 'req-a2ed5bde-9982-4357-b737-50b832738db8'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:20.299928 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-a2ed5bde-9982-4357-b737-50b832738db8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11267524c7534eab8ede077f99487031', 'request_id': 'req-a2ed5bde-9982-4357-b737-50b832738db8'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:20.309567 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-a2ed5bde-9982-4357-b737-50b832738db8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11267524c7534eab8ede077f99487031', 'request_id': 'req-a2ed5bde-9982-4357-b737-50b832738db8'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:20.322554 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-a2ed5bde-9982-4357-b737-50b832738db8 admin admin] The token cache is being invalidate because project 11267524c7534eab8ede077f99487031 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:20.322876 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-a2ed5bde-9982-4357-b737-50b832738db8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:20.323551 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-a2ed5bde-9982-4357-b737-50b832738db8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:20.325169 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1025/2042] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:20 2022] DELETE /identity/v3/projects/11267524c7534eab8ede077f99487031 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:20.327683 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ed5bde-9982-4357-b737-50b832738db8 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:20.330051 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-a2ed5bde-9982-4357-b737-50b832738db8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:20.354310 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1018/2043] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:20.357938 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809569dc-cc95-41d7-9d3f-4855509dcb6a None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:20.357938 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809569dc-cc95-41d7-9d3f-4855509dcb6a None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:20.357938 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809569dc-cc95-41d7-9d3f-4855509dcb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:20.377081 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-809569dc-cc95-41d7-9d3f-4855509dcb6a None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:20.378475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-809569dc-cc95-41d7-9d3f-4855509dcb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:20.380574 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:20.387641 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1019/2044] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:20.388520 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809569dc-cc95-41d7-9d3f-4855509dcb6a None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:20.398195 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809569dc-cc95-41d7-9d3f-4855509dcb6a None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:20.402825 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31b234-dca9-42ba-8718-26c86e0a0d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:20.403430 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31b234-dca9-42ba-8718-26c86e0a0d20 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:20.403775 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31b234-dca9-42ba-8718-26c86e0a0d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:20.403985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31b234-dca9-42ba-8718-26c86e0a0d20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:20.404619 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af31b234-dca9-42ba-8718-26c86e0a0d20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:30:20.404913 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af31b234-dca9-42ba-8718-26c86e0a0d20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:20.405025 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ed5bde-9982-4357-b737-50b832738db8 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:20.406536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af31b234-dca9-42ba-8718-26c86e0a0d20 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:20.410103 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da8ee9-7936-4f69-b89b-a5861322f7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:20.410741 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da8ee9-7936-4f69-b89b-a5861322f7fc admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:20.411021 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da8ee9-7936-4f69-b89b-a5861322f7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:20.411288 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da8ee9-7936-4f69-b89b-a5861322f7fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:20.411923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da8ee9-7936-4f69-b89b-a5861322f7fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:20.413963 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da8ee9-7936-4f69-b89b-a5861322f7fc admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:20.423413 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1020/2045] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:30:20.425971 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e0c42-474d-4d59-9a16-2fc29acde750 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:20.426382 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e0c42-474d-4d59-9a16-2fc29acde750 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:20.426885 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e0c42-474d-4d59-9a16-2fc29acde750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:20.433271 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1026/2046] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:20.434106 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da8ee9-7936-4f69-b89b-a5861322f7fc admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:20.447014 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da8ee9-7936-4f69-b89b-a5861322f7fc admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:20.451709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-e13e0c42-474d-4d59-9a16-2fc29acde750 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:20.451949 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da3d99-a6c9-4f04-b39f-772316b7c692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:20.452557 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da3d99-a6c9-4f04-b39f-772316b7c692 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:20.452830 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da3d99-a6c9-4f04-b39f-772316b7c692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:20.452944 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-e13e0c42-474d-4d59-9a16-2fc29acde750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:20.453099 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da3d99-a6c9-4f04-b39f-772316b7c692 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:20.454111 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2da3d99-a6c9-4f04-b39f-772316b7c692 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:20.456405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2da3d99-a6c9-4f04-b39f-772316b7c692 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:20.461651 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1021/2047] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:20.463031 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13e0c42-474d-4d59-9a16-2fc29acde750 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:20.472512 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13e0c42-474d-4d59-9a16-2fc29acde750 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:20.477273 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce647d-a57b-4761-b206-97cb6dc068c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:20.477874 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce647d-a57b-4761-b206-97cb6dc068c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:20.478148 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce647d-a57b-4761-b206-97cb6dc068c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:20.478413 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce647d-a57b-4761-b206-97cb6dc068c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:20.479231 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ce647d-a57b-4761-b206-97cb6dc068c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:20.480829 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ce647d-a57b-4761-b206-97cb6dc068c8 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:20.493569 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1022/2048] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:20.494366 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce647d-a57b-4761-b206-97cb6dc068c8 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:20.499806 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1027/2049] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:20 2022] POST /identity/v3/projects => generated 379 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:20.501778 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da3d99-a6c9-4f04-b39f-772316b7c692 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:20.508690 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce647d-a57b-4761-b206-97cb6dc068c8 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:20.588820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da3d99-a6c9-4f04-b39f-772316b7c692 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:20.603745 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221939c8-5b65-4462-a1a0-2ce8894f5db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:20.604172 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184eb841-24f1-42a3-bc9f-bd60b4493262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:20.604799 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221939c8-5b65-4462-a1a0-2ce8894f5db0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:20.605309 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221939c8-5b65-4462-a1a0-2ce8894f5db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:20.605677 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221939c8-5b65-4462-a1a0-2ce8894f5db0 admin admin] PATH_INFO: `/v3/projects/8aca24dbea0247549df7bdc687f7e2a7/users/2ffdd4e961244bffa2b22490956ab51f/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:20.606948 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184eb841-24f1-42a3-bc9f-bd60b4493262 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:20.607515 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184eb841-24f1-42a3-bc9f-bd60b4493262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:20.607932 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184eb841-24f1-42a3-bc9f-bd60b4493262 admin admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:20.619542 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184eb841-24f1-42a3-bc9f-bd60b4493262 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:30:20.619542 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184eb841-24f1-42a3-bc9f-bd60b4493262 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:20.622880 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184eb841-24f1-42a3-bc9f-bd60b4493262 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:20.627675 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221939c8-5b65-4462-a1a0-2ce8894f5db0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aca24dbea0247549df7bdc687f7e2a7, user_id=2ffdd4e961244bffa2b22490956ab51f, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:20.630553 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221939c8-5b65-4462-a1a0-2ce8894f5db0 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:20.637672 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1023/2050] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:30:20.639668 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184eb841-24f1-42a3-bc9f-bd60b4493262 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:20.645468 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1028/2051] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:20 2022] PUT /identity/v3/projects/8aca24dbea0247549df7bdc687f7e2a7/users/2ffdd4e961244bffa2b22490956ab51f/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:20.646430 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221939c8-5b65-4462-a1a0-2ce8894f5db0 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:20.653479 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184eb841-24f1-42a3-bc9f-bd60b4493262 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:20.718764 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:20.726825 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11b202a-f36a-4bd3-aa26-a276ccc86207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:20.727361 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b202a-f36a-4bd3-aa26-a276ccc86207 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:20.727547 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221939c8-5b65-4462-a1a0-2ce8894f5db0 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:20.727657 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b202a-f36a-4bd3-aa26-a276ccc86207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:20.727887 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b202a-f36a-4bd3-aa26-a276ccc86207 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:20.728517 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11b202a-f36a-4bd3-aa26-a276ccc86207 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:20.730826 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11b202a-f36a-4bd3-aa26-a276ccc86207 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:20.733053 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80d8b1-f1d3-4518-9468-8d4dbee7688b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:20.734875 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80d8b1-f1d3-4518-9468-8d4dbee7688b admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:20.734875 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80d8b1-f1d3-4518-9468-8d4dbee7688b admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:20.734875 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80d8b1-f1d3-4518-9468-8d4dbee7688b admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:20.737070 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa80d8b1-f1d3-4518-9468-8d4dbee7688b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:20.739228 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa80d8b1-f1d3-4518-9468-8d4dbee7688b admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:20.763042 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1024/2052] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:20 2022] POST /identity/v3/users => generated 290 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:20.763042 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c783475-f68d-4066-8618-2b79cc805dca None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:20.763042 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c783475-f68d-4066-8618-2b79cc805dca None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:20.763042 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c783475-f68d-4066-8618-2b79cc805dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:20.768954 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1029/2053] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:20 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:20.769673 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80d8b1-f1d3-4518-9468-8d4dbee7688b admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:20.788277 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-6c783475-f68d-4066-8618-2b79cc805dca None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:20.789768 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-6c783475-f68d-4066-8618-2b79cc805dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:20.799132 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:20.839662 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80d8b1-f1d3-4518-9468-8d4dbee7688b admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:20.844568 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c2b47-2d74-4d55-b2ba-cf29365c4161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:20.845429 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c2b47-2d74-4d55-b2ba-cf29365c4161 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:20.845715 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c2b47-2d74-4d55-b2ba-cf29365c4161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:20.846090 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c2b47-2d74-4d55-b2ba-cf29365c4161 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:20.846719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c2b47-2d74-4d55-b2ba-cf29365c4161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:20.848576 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c2b47-2d74-4d55-b2ba-cf29365c4161 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:20.859570 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1025/2054] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:20.862891 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c783475-f68d-4066-8618-2b79cc805dca None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:20.868675 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1030/2055] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:20.869763 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c2b47-2d74-4d55-b2ba-cf29365c4161 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:20.869976 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c783475-f68d-4066-8618-2b79cc805dca None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:20.874610 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf1d96-f9e8-4475-a29f-b58760ddfe32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:20.875317 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf1d96-f9e8-4475-a29f-b58760ddfe32 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:20.875606 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf1d96-f9e8-4475-a29f-b58760ddfe32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:20.876036 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf1d96-f9e8-4475-a29f-b58760ddfe32 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:20.877184 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cf1d96-f9e8-4475-a29f-b58760ddfe32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:20.879176 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cf1d96-f9e8-4475-a29f-b58760ddfe32 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:20.880447 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c2b47-2d74-4d55-b2ba-cf29365c4161 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:20.885094 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fc50c-7293-4ecb-ad90-ca8ab2183e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:20.885691 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fc50c-7293-4ecb-ad90-ca8ab2183e0c admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:20.885967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fc50c-7293-4ecb-ad90-ca8ab2183e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:20.886299 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fc50c-7293-4ecb-ad90-ca8ab2183e0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:20.887008 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520fc50c-7293-4ecb-ad90-ca8ab2183e0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:20.889034 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520fc50c-7293-4ecb-ad90-ca8ab2183e0c admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:20.904038 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1031/2056] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:20.904920 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fc50c-7293-4ecb-ad90-ca8ab2183e0c admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:20.907306 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-520fc50c-7293-4ecb-ad90-ca8ab2183e0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:20.907555 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1026/2057] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:20 2022] POST /identity/v3/users => generated 287 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:20.908521 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf1d96-f9e8-4475-a29f-b58760ddfe32 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:20.918948 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf1d96-f9e8-4475-a29f-b58760ddfe32 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:20.924381 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a5491c-784b-4343-b780-74a535da668f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:20.925111 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a5491c-784b-4343-b780-74a535da668f admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:20.925389 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a5491c-784b-4343-b780-74a535da668f admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:20.925661 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a5491c-784b-4343-b780-74a535da668f admin admin] PATH_INFO: `/v3/projects/9defab9d20434d3b9e7d04e32fccc49d/users/0fe86af4087c4fe5900cd659823f0af8/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:20.943356 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a5491c-784b-4343-b780-74a535da668f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9defab9d20434d3b9e7d04e32fccc49d, user_id=0fe86af4087c4fe5900cd659823f0af8, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:20.946202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a5491c-784b-4343-b780-74a535da668f admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:20.978303 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1027/2058] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:20 2022] PUT /identity/v3/projects/9defab9d20434d3b9e7d04e32fccc49d/users/0fe86af4087c4fe5900cd659823f0af8/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:20.978979 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a5491c-784b-4343-b780-74a535da668f admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:20.984527 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fc50c-7293-4ecb-ad90-ca8ab2183e0c admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:21.048759 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d7a44-96d1-4cd4-b567-ae5e1ec59289 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:21.049577 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d7a44-96d1-4cd4-b567-ae5e1ec59289 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:21.049998 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d7a44-96d1-4cd4-b567-ae5e1ec59289 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:21.050369 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d7a44-96d1-4cd4-b567-ae5e1ec59289 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:21.063958 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a5491c-784b-4343-b780-74a535da668f admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:21.068651 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63be177c-adbe-40d9-a6f7-c86255d95182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:21.069378 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be177c-adbe-40d9-a6f7-c86255d95182 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:21.069658 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be177c-adbe-40d9-a6f7-c86255d95182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:21.069924 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be177c-adbe-40d9-a6f7-c86255d95182 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:21.070565 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63be177c-adbe-40d9-a6f7-c86255d95182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:21.072205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63be177c-adbe-40d9-a6f7-c86255d95182 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:21.082531 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1028/2059] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:21.083988 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266629e-8040-4952-8a5a-051c2448b62a None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:21.084268 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266629e-8040-4952-8a5a-051c2448b62a None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:21.084536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266629e-8040-4952-8a5a-051c2448b62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:21.120996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-e266629e-8040-4952-8a5a-051c2448b62a None None] MFA Rules not processed for user `0fe86af4087c4fe5900cd659823f0af8`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:21.122220 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-e266629e-8040-4952-8a5a-051c2448b62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:21.122374 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d7a44-96d1-4cd4-b567-ae5e1ec59289 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:21.124009 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d7a44-96d1-4cd4-b567-ae5e1ec59289 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:21.134331 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1032/2060] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:21.135258 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d7a44-96d1-4cd4-b567-ae5e1ec59289 service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:21.147102 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d7a44-96d1-4cd4-b567-ae5e1ec59289 service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:21.152425 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c4f0cb-bdea-43d5-91c8-0cb44e0bd7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:21.153054 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c4f0cb-bdea-43d5-91c8-0cb44e0bd7a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:21.153728 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c4f0cb-bdea-43d5-91c8-0cb44e0bd7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:21.153728 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c4f0cb-bdea-43d5-91c8-0cb44e0bd7a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:21.154408 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c4f0cb-bdea-43d5-91c8-0cb44e0bd7a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:21.156467 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c4f0cb-bdea-43d5-91c8-0cb44e0bd7a4 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:21.160153 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:21.174079 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1033/2061] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:21.179800 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c4f0cb-bdea-43d5-91c8-0cb44e0bd7a4 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:21.188592 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:21.196173 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c4f0cb-bdea-43d5-91c8-0cb44e0bd7a4 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:21.204226 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0363683-5e17-497c-af5e-8c388ab47f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:21.205056 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0363683-5e17-497c-af5e-8c388ab47f83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:21.205401 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0363683-5e17-497c-af5e-8c388ab47f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:21.205773 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0363683-5e17-497c-af5e-8c388ab47f83 admin admin] PATH_INFO: `/v3/projects/1eab4b25d02147ef8232c5d37290005c/users/3f9fe0442c2b41fa822780b1b7b3b807/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:21.218704 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1029/2062] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:21.218704 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e266629e-8040-4952-8a5a-051c2448b62a None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:21.231798 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e266629e-8040-4952-8a5a-051c2448b62a None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:21.236501 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0363683-5e17-497c-af5e-8c388ab47f83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eab4b25d02147ef8232c5d37290005c, user_id=3f9fe0442c2b41fa822780b1b7b3b807, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:21.236933 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa891c0-e8eb-4443-bd85-bf306568ff76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:21.237687 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa891c0-e8eb-4443-bd85-bf306568ff76 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:21.237966 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa891c0-e8eb-4443-bd85-bf306568ff76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:21.238326 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa891c0-e8eb-4443-bd85-bf306568ff76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:21.239418 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0363683-5e17-497c-af5e-8c388ab47f83 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:21.242293 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa891c0-e8eb-4443-bd85-bf306568ff76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:21.244681 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa891c0-e8eb-4443-bd85-bf306568ff76 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:21.263887 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1034/2063] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:21 2022] PUT /identity/v3/projects/1eab4b25d02147ef8232c5d37290005c/users/3f9fe0442c2b41fa822780b1b7b3b807/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:21.271280 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5bc122-f52b-496a-8dbc-c29681ef2aef None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:21.271551 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5bc122-f52b-496a-8dbc-c29681ef2aef None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:21.272238 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5bc122-f52b-496a-8dbc-c29681ef2aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:21.296920 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1030/2064] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:21 2022] POST /identity/v3/projects => generated 377 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:21.298610 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa891c0-e8eb-4443-bd85-bf306568ff76 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:21.301176 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-3d5bc122-f52b-496a-8dbc-c29681ef2aef None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:21.302538 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-3d5bc122-f52b-496a-8dbc-c29681ef2aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:21.391809 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa891c0-e8eb-4443-bd85-bf306568ff76 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:21.397854 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a2078f-1eb6-4696-a373-e31e776290d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:21.398812 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2078f-1eb6-4696-a373-e31e776290d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:21.399191 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2078f-1eb6-4696-a373-e31e776290d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:21.399572 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2078f-1eb6-4696-a373-e31e776290d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:21.400400 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a2078f-1eb6-4696-a373-e31e776290d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:21.402619 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a2078f-1eb6-4696-a373-e31e776290d7 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:21.412892 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1035/2065] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:21.440013 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1031/2066] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:21 2022] POST /identity/v3/users => generated 289 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:21.441306 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5bc122-f52b-496a-8dbc-c29681ef2aef None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:21.450770 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a2078f-1eb6-4696-a373-e31e776290d7 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:21.453690 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5bc122-f52b-496a-8dbc-c29681ef2aef None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:21.471175 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798791c7-ae4b-4e19-919a-97eb9b218635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:21.471784 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798791c7-ae4b-4e19-919a-97eb9b218635 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:21.472064 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798791c7-ae4b-4e19-919a-97eb9b218635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:21.472327 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798791c7-ae4b-4e19-919a-97eb9b218635 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:21.473059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798791c7-ae4b-4e19-919a-97eb9b218635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:21.474832 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798791c7-ae4b-4e19-919a-97eb9b218635 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:21.503019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1036/2067] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:21.504208 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798791c7-ae4b-4e19-919a-97eb9b218635 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:21.524985 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798791c7-ae4b-4e19-919a-97eb9b218635 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:21.533185 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64990d7e-0669-426d-9b39-a07cfba23f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:21.534720 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64990d7e-0669-426d-9b39-a07cfba23f36 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:21.535265 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64990d7e-0669-426d-9b39-a07cfba23f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:21.536019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64990d7e-0669-426d-9b39-a07cfba23f36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:21.537231 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64990d7e-0669-426d-9b39-a07cfba23f36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:21.539817 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64990d7e-0669-426d-9b39-a07cfba23f36 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:21.556305 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1037/2068] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:21.557483 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64990d7e-0669-426d-9b39-a07cfba23f36 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:21.560209 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a2078f-1eb6-4696-a373-e31e776290d7 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:21.564635 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b33d5-6eaa-4c6b-947b-6a585d62232d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:21.565686 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b33d5-6eaa-4c6b-947b-6a585d62232d service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:21.565992 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b33d5-6eaa-4c6b-947b-6a585d62232d service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:21.567263 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b33d5-6eaa-4c6b-947b-6a585d62232d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:21.573888 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64990d7e-0669-426d-9b39-a07cfba23f36 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:21.579897 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1082b37-81d1-4708-a24e-932b972f9934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:21.581008 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1082b37-81d1-4708-a24e-932b972f9934 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:21.581498 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1082b37-81d1-4708-a24e-932b972f9934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:21.582302 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1082b37-81d1-4708-a24e-932b972f9934 admin admin] PATH_INFO: `/v3/projects/c8f41f8305434d76ab3dc9023c6c4025/users/0dcc020bd6e0419491e92526525366f4/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:21.584283 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b33d5-6eaa-4c6b-947b-6a585d62232d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:21.586123 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b33d5-6eaa-4c6b-947b-6a585d62232d service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:21.603497 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1082b37-81d1-4708-a24e-932b972f9934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8f41f8305434d76ab3dc9023c6c4025, user_id=0dcc020bd6e0419491e92526525366f4, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:21.603812 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1032/2069] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:21.606904 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1082b37-81d1-4708-a24e-932b972f9934 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:21.642918 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1038/2070] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:21 2022] PUT /identity/v3/projects/c8f41f8305434d76ab3dc9023c6c4025/users/0dcc020bd6e0419491e92526525366f4/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:21.643212 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d7ea3-7ed2-4c5a-a867-32c349dabfe3 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:21.643552 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d7ea3-7ed2-4c5a-a867-32c349dabfe3 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:21.643831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d7ea3-7ed2-4c5a-a867-32c349dabfe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:21.700658 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-af2d7ea3-7ed2-4c5a-a867-32c349dabfe3 None None] MFA Rules not processed for user `0dcc020bd6e0419491e92526525366f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:21.701881 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-af2d7ea3-7ed2-4c5a-a867-32c349dabfe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:21.752516 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:21.823108 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1033/2071] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:21.827762 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1082b37-81d1-4708-a24e-932b972f9934 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:21.973192 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1082b37-81d1-4708-a24e-932b972f9934 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:21.995531 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8ba26f-586d-4360-b57b-d67a8b286542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:21.996268 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ba26f-586d-4360-b57b-d67a8b286542 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:21.996988 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ba26f-586d-4360-b57b-d67a8b286542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:21.996988 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ba26f-586d-4360-b57b-d67a8b286542 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:21.997678 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8ba26f-586d-4360-b57b-d67a8b286542 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:21.999900 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8ba26f-586d-4360-b57b-d67a8b286542 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:22.055677 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1039/2072] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:21 2022] POST /identity/v3/projects => generated 379 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:22.057235 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d7ea3-7ed2-4c5a-a867-32c349dabfe3 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:22.070938 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d7ea3-7ed2-4c5a-a867-32c349dabfe3 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:22.138337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e740f-2654-4d27-9926-7f8efea28010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:22.139420 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e740f-2654-4d27-9926-7f8efea28010 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:22.139935 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e740f-2654-4d27-9926-7f8efea28010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:22.140309 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e740f-2654-4d27-9926-7f8efea28010 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:22.141082 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e740f-2654-4d27-9926-7f8efea28010 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:22.143249 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e740f-2654-4d27-9926-7f8efea28010 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:22.173500 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1034/2073] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:22 2022] POST /identity/v3/users => generated 296 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:30:22.177712 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8ba26f-586d-4360-b57b-d67a8b286542 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:22.189174 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8ba26f-586d-4360-b57b-d67a8b286542 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:22.193918 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a65b9-3d38-44e7-a6e0-fea4eb14a28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:22.194548 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a65b9-3d38-44e7-a6e0-fea4eb14a28e admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:22.195412 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a65b9-3d38-44e7-a6e0-fea4eb14a28e admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:22.195412 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a65b9-3d38-44e7-a6e0-fea4eb14a28e admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:22.195751 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a65b9-3d38-44e7-a6e0-fea4eb14a28e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:22.197380 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a65b9-3d38-44e7-a6e0-fea4eb14a28e admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:22.214273 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e740f-2654-4d27-9926-7f8efea28010 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:22.225446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1040/2074] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:22.237156 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e740f-2654-4d27-9926-7f8efea28010 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:22.258684 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66450591-6113-4ab0-b73c-423a8d899495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:22.258684 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66450591-6113-4ab0-b73c-423a8d899495 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:22.258684 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66450591-6113-4ab0-b73c-423a8d899495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:22.259427 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66450591-6113-4ab0-b73c-423a8d899495 admin admin] PATH_INFO: `/v3/projects/826c9c6710ea443ab66a9409f2fbde77/users/b957d44949dd4272859dcb9c573d14ec/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:22.311607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66450591-6113-4ab0-b73c-423a8d899495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=826c9c6710ea443ab66a9409f2fbde77, user_id=b957d44949dd4272859dcb9c573d14ec, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:22.314122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22f223-ee81-4a85-a445-b9b24aac4fe8 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:22.314640 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22f223-ee81-4a85-a445-b9b24aac4fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:22.315033 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22f223-ee81-4a85-a445-b9b24aac4fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:22.315991 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66450591-6113-4ab0-b73c-423a8d899495 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:22.336507 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1035/2075] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:22 2022] PUT /identity/v3/projects/826c9c6710ea443ab66a9409f2fbde77/users/b957d44949dd4272859dcb9c573d14ec/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:22.342338 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66450591-6113-4ab0-b73c-423a8d899495 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:22.350721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:22.367099 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-1e22f223-ee81-4a85-a445-b9b24aac4fe8 None None] MFA Rules not processed for user `2ffdd4e961244bffa2b22490956ab51f`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:22.370474 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-1e22f223-ee81-4a85-a445-b9b24aac4fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:22.463950 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66450591-6113-4ab0-b73c-423a8d899495 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:22.469370 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a2185d-b91c-4215-a600-54d6ee791427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:22.470088 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2185d-b91c-4215-a600-54d6ee791427 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:22.470435 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2185d-b91c-4215-a600-54d6ee791427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:22.470815 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2185d-b91c-4215-a600-54d6ee791427 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:22.471656 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a2185d-b91c-4215-a600-54d6ee791427 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:22.473473 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a2185d-b91c-4215-a600-54d6ee791427 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:22.478877 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1041/2076] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:22.489302 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e22f223-ee81-4a85-a445-b9b24aac4fe8 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:22.489558 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1036/2077] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:22.489660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-1e22f223-ee81-4a85-a445-b9b24aac4fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:22.491525 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a2185d-b91c-4215-a600-54d6ee791427 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:22.513948 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a2185d-b91c-4215-a600-54d6ee791427 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:22.533815 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f21292-9ea3-4ead-a953-8a6dc8817b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:22.533815 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f21292-9ea3-4ead-a953-8a6dc8817b6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:22.533815 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f21292-9ea3-4ead-a953-8a6dc8817b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:22.533815 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f21292-9ea3-4ead-a953-8a6dc8817b6c admin admin] PATH_INFO: `/v3/projects/826c9c6710ea443ab66a9409f2fbde77/users/b957d44949dd4272859dcb9c573d14ec/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:22.535627 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f21292-9ea3-4ead-a953-8a6dc8817b6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=826c9c6710ea443ab66a9409f2fbde77, user_id=b957d44949dd4272859dcb9c573d14ec, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:22.538421 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f21292-9ea3-4ead-a953-8a6dc8817b6c admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:22.564600 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1037/2078] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:22 2022] PUT /identity/v3/projects/826c9c6710ea443ab66a9409f2fbde77/users/b957d44949dd4272859dcb9c573d14ec/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:22.576236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f4125-3f1b-427d-944f-790223ab4f10 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:22.576701 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f4125-3f1b-427d-944f-790223ab4f10 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:22.577094 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f4125-3f1b-427d-944f-790223ab4f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:22.625213 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e22f223-ee81-4a85-a445-b9b24aac4fe8 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:22.640723 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-e46f4125-3f1b-427d-944f-790223ab4f10 None None] MFA Rules not processed for user `b957d44949dd4272859dcb9c573d14ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:22.642418 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-e46f4125-3f1b-427d-944f-790223ab4f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:22.655693 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:22.756046 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb86889-eee3-4eb3-a4fc-5dfea44d18a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:22.758069 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb86889-eee3-4eb3-a4fc-5dfea44d18a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:22.758366 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb86889-eee3-4eb3-a4fc-5dfea44d18a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:22.758635 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb86889-eee3-4eb3-a4fc-5dfea44d18a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:22.776867 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1038/2079] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:22 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:30:22.788664 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f4125-3f1b-427d-944f-790223ab4f10 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:22.790272 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-e46f4125-3f1b-427d-944f-790223ab4f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:22.804255 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f4125-3f1b-427d-944f-790223ab4f10 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:22.808022 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a952a5-5f58-45c7-bed7-2272fcb68bf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:22.808643 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a952a5-5f58-45c7-bed7-2272fcb68bf9 service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:22.808915 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a952a5-5f58-45c7-bed7-2272fcb68bf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:22.809495 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a952a5-5f58-45c7-bed7-2272fcb68bf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:22.839797 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a952a5-5f58-45c7-bed7-2272fcb68bf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:22.841254 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a952a5-5f58-45c7-bed7-2272fcb68bf9 service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:22.855969 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1039/2080] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:22 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:30:22.868862 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb86889-eee3-4eb3-a4fc-5dfea44d18a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:22.871462 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb86889-eee3-4eb3-a4fc-5dfea44d18a8 service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:22.884477 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1042/2081] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:22.985889 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a952a5-5f58-45c7-bed7-2272fcb68bf9 service cinder] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:22.995258 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a952a5-5f58-45c7-bed7-2272fcb68bf9 service cinder] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:22.999616 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac68065-40c8-4a48-9d0f-9598eab534a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:23.000516 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac68065-40c8-4a48-9d0f-9598eab534a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:23.000986 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac68065-40c8-4a48-9d0f-9598eab534a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:23.001796 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac68065-40c8-4a48-9d0f-9598eab534a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:23.071444 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac68065-40c8-4a48-9d0f-9598eab534a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:23.073061 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac68065-40c8-4a48-9d0f-9598eab534a8 service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:23.084976 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1040/2082] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:22 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:24.564386 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f559d3b-e3dd-45dc-b900-f55fd2cf41e6 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:24.564876 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f559d3b-e3dd-45dc-b900-f55fd2cf41e6 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:24.564970 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f559d3b-e3dd-45dc-b900-f55fd2cf41e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:24.586925 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-7f559d3b-e3dd-45dc-b900-f55fd2cf41e6 None None] MFA Rules not processed for user `2ffdd4e961244bffa2b22490956ab51f`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:24.588139 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-7f559d3b-e3dd-45dc-b900-f55fd2cf41e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:24.606571 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1043/2083] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:26.429227 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac68065-40c8-4a48-9d0f-9598eab534a8 service cinder] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:26.497023 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac68065-40c8-4a48-9d0f-9598eab534a8 service cinder] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:26.503156 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee397d2c-e01c-4402-9b94-47359c223cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:26.504126 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee397d2c-e01c-4402-9b94-47359c223cc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:26.504593 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee397d2c-e01c-4402-9b94-47359c223cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:26.505010 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee397d2c-e01c-4402-9b94-47359c223cc3 admin admin] PATH_INFO: `/v3/users/0fe86af4087c4fe5900cd659823f0af8` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:26.506440 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee397d2c-e01c-4402-9b94-47359c223cc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fe86af4087c4fe5900cd659823f0af8)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:26.508651 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee397d2c-e01c-4402-9b94-47359c223cc3 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:26.569380 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:26.610617 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-ee397d2c-e01c-4402-9b94-47359c223cc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fe86af4087c4fe5900cd659823f0af8'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:26.612581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-ee397d2c-e01c-4402-9b94-47359c223cc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fe86af4087c4fe5900cd659823f0af8'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:26.629818 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-ee397d2c-e01c-4402-9b94-47359c223cc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fe86af4087c4fe5900cd659823f0af8'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:26.652114 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-ee397d2c-e01c-4402-9b94-47359c223cc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fe86af4087c4fe5900cd659823f0af8'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:26.689678 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-ee397d2c-e01c-4402-9b94-47359c223cc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fe86af4087c4fe5900cd659823f0af8'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:26.707042 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1041/2084] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:26 2022] DELETE /identity/v3/users/0fe86af4087c4fe5900cd659823f0af8 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:26.711523 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823be059-a4a1-40c8-bdf1-098938ce752a None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:26.713529 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823be059-a4a1-40c8-bdf1-098938ce752a None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:26.714649 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823be059-a4a1-40c8-bdf1-098938ce752a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:26.728345 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:26.759492 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-823be059-a4a1-40c8-bdf1-098938ce752a None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:26.763222 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-823be059-a4a1-40c8-bdf1-098938ce752a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:26.914264 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1044/2085] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:30:26.935523 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee397d2c-e01c-4402-9b94-47359c223cc3 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:26.938537 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-ee397d2c-e01c-4402-9b94-47359c223cc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:27.107725 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee397d2c-e01c-4402-9b94-47359c223cc3 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:27.120530 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d0d21-6864-47c6-a71f-06cf2d7847bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:27.121175 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d0d21-6864-47c6-a71f-06cf2d7847bc service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:27.121451 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d0d21-6864-47c6-a71f-06cf2d7847bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:27.121720 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d0d21-6864-47c6-a71f-06cf2d7847bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:27.142678 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13d0d21-6864-47c6-a71f-06cf2d7847bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:27.143692 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13d0d21-6864-47c6-a71f-06cf2d7847bc service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:27.173192 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1042/2086] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:28.196956 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823be059-a4a1-40c8-bdf1-098938ce752a None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:28.197614 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56bda18-2829-46c5-b681-d3c4c0246d1f None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:28.198008 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56bda18-2829-46c5-b681-d3c4c0246d1f None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:28.198154 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56bda18-2829-46c5-b681-d3c4c0246d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:28.198791 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-823be059-a4a1-40c8-bdf1-098938ce752a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:28.215753 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823be059-a4a1-40c8-bdf1-098938ce752a None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:28.222013 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e9ee60-2c81-459f-82ab-c47a60eef409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:28.223323 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e9ee60-2c81-459f-82ab-c47a60eef409 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:28.223807 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e9ee60-2c81-459f-82ab-c47a60eef409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:28.224281 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e9ee60-2c81-459f-82ab-c47a60eef409 admin admin] PATH_INFO: `/v3/projects/9defab9d20434d3b9e7d04e32fccc49d` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:28.225728 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e9ee60-2c81-459f-82ab-c47a60eef409 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9defab9d20434d3b9e7d04e32fccc49d)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:28.228190 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e9ee60-2c81-459f-82ab-c47a60eef409 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:28.272032 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-b56bda18-2829-46c5-b681-d3c4c0246d1f None None] MFA Rules not processed for user `be586640013949589d2a7b11a7a2f3b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:28.277059 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-b56bda18-2829-46c5-b681-d3c4c0246d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:28.325935 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-e9e9ee60-2c81-459f-82ab-c47a60eef409 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9defab9d20434d3b9e7d04e32fccc49d', 'request_id': 'req-e9e9ee60-2c81-459f-82ab-c47a60eef409'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:28.349105 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-e9e9ee60-2c81-459f-82ab-c47a60eef409 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9defab9d20434d3b9e7d04e32fccc49d', 'request_id': 'req-e9e9ee60-2c81-459f-82ab-c47a60eef409'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:28.362304 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-e9e9ee60-2c81-459f-82ab-c47a60eef409 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9defab9d20434d3b9e7d04e32fccc49d', 'request_id': 'req-e9e9ee60-2c81-459f-82ab-c47a60eef409'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:28.403143 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-e9e9ee60-2c81-459f-82ab-c47a60eef409 admin admin] The token cache is being invalidate because project 9defab9d20434d3b9e7d04e32fccc49d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:28.403489 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-e9e9ee60-2c81-459f-82ab-c47a60eef409 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:28.404172 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-e9e9ee60-2c81-459f-82ab-c47a60eef409 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:28.407031 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1045/2087] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:28 2022] DELETE /identity/v3/projects/9defab9d20434d3b9e7d04e32fccc49d => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:28.412744 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e9ee60-2c81-459f-82ab-c47a60eef409 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:28.414625 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-e9e9ee60-2c81-459f-82ab-c47a60eef409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:28.457271 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1043/2088] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:28.539658 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e9ee60-2c81-459f-82ab-c47a60eef409 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:28.549225 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5832f1-d82e-40c1-a50c-8b9375959004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:28.549896 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5832f1-d82e-40c1-a50c-8b9375959004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:28.550167 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5832f1-d82e-40c1-a50c-8b9375959004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:28.550437 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5832f1-d82e-40c1-a50c-8b9375959004 admin admin] PATH_INFO: `/v3/users/0dcc020bd6e0419491e92526525366f4` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:28.551664 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5832f1-d82e-40c1-a50c-8b9375959004 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dcc020bd6e0419491e92526525366f4)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:28.553638 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5832f1-d82e-40c1-a50c-8b9375959004 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:28.604338 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:28.655244 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-ae5832f1-d82e-40c1-a50c-8b9375959004 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dcc020bd6e0419491e92526525366f4'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:28.667804 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-ae5832f1-d82e-40c1-a50c-8b9375959004 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dcc020bd6e0419491e92526525366f4'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:28.668736 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-ae5832f1-d82e-40c1-a50c-8b9375959004 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dcc020bd6e0419491e92526525366f4'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:28.689194 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-ae5832f1-d82e-40c1-a50c-8b9375959004 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dcc020bd6e0419491e92526525366f4'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:28.701657 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-ae5832f1-d82e-40c1-a50c-8b9375959004 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dcc020bd6e0419491e92526525366f4'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:28.713135 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1046/2089] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:28 2022] DELETE /identity/v3/users/0dcc020bd6e0419491e92526525366f4 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:28.779735 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb08a4f-8c68-4955-ab57-889487dce03c None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:28.780332 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb08a4f-8c68-4955-ab57-889487dce03c None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:28.780917 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb08a4f-8c68-4955-ab57-889487dce03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:28.837697 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-beb08a4f-8c68-4955-ab57-889487dce03c None None] MFA Rules not processed for user `3f9fe0442c2b41fa822780b1b7b3b807`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:28.847438 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-beb08a4f-8c68-4955-ab57-889487dce03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:28.932703 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:29.003917 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1044/2090] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:29.008438 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5832f1-d82e-40c1-a50c-8b9375959004 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:29.009874 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-ae5832f1-d82e-40c1-a50c-8b9375959004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:29.096963 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5832f1-d82e-40c1-a50c-8b9375959004 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:29.102152 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53a714-719b-44f5-b5ef-a82dc2a8bcea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:29.102940 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53a714-719b-44f5-b5ef-a82dc2a8bcea admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:29.103353 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53a714-719b-44f5-b5ef-a82dc2a8bcea admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:29.103638 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53a714-719b-44f5-b5ef-a82dc2a8bcea admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:29.106916 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b53a714-719b-44f5-b5ef-a82dc2a8bcea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:29.106916 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b53a714-719b-44f5-b5ef-a82dc2a8bcea admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:29.109641 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb08a4f-8c68-4955-ab57-889487dce03c None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:29.110235 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-beb08a4f-8c68-4955-ab57-889487dce03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:29.126629 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb08a4f-8c68-4955-ab57-889487dce03c None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:29.132176 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c64ac10-37e8-4aad-bb4a-af313535f38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:29.132954 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c64ac10-37e8-4aad-bb4a-af313535f38f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:29.133344 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c64ac10-37e8-4aad-bb4a-af313535f38f admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:29.133741 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c64ac10-37e8-4aad-bb4a-af313535f38f admin admin] PATH_INFO: `/v3/projects/c8f41f8305434d76ab3dc9023c6c4025` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:29.135053 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c64ac10-37e8-4aad-bb4a-af313535f38f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8f41f8305434d76ab3dc9023c6c4025)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:29.138755 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c64ac10-37e8-4aad-bb4a-af313535f38f admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:29.175283 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1047/2091] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:29 2022] POST /identity/v3/projects => generated 373 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:29.179346 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53a714-719b-44f5-b5ef-a82dc2a8bcea admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:29.208871 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-9c64ac10-37e8-4aad-bb4a-af313535f38f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8f41f8305434d76ab3dc9023c6c4025', 'request_id': 'req-9c64ac10-37e8-4aad-bb4a-af313535f38f'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:29.216547 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-9c64ac10-37e8-4aad-bb4a-af313535f38f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8f41f8305434d76ab3dc9023c6c4025', 'request_id': 'req-9c64ac10-37e8-4aad-bb4a-af313535f38f'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:29.223537 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-9c64ac10-37e8-4aad-bb4a-af313535f38f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8f41f8305434d76ab3dc9023c6c4025', 'request_id': 'req-9c64ac10-37e8-4aad-bb4a-af313535f38f'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:29.241503 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-9c64ac10-37e8-4aad-bb4a-af313535f38f admin admin] The token cache is being invalidate because project c8f41f8305434d76ab3dc9023c6c4025 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:29.241797 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-9c64ac10-37e8-4aad-bb4a-af313535f38f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:29.242484 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-9c64ac10-37e8-4aad-bb4a-af313535f38f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:29.247268 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1045/2092] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:29 2022] DELETE /identity/v3/projects/c8f41f8305434d76ab3dc9023c6c4025 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:29.248351 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c64ac10-37e8-4aad-bb4a-af313535f38f admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:29.249945 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-9c64ac10-37e8-4aad-bb4a-af313535f38f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:29.273843 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53a714-719b-44f5-b5ef-a82dc2a8bcea admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:29.313321 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:29.337762 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c64ac10-37e8-4aad-bb4a-af313535f38f admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:29.343543 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81663e-9e3d-4ae0-ab34-e1da383cae48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:29.344244 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81663e-9e3d-4ae0-ab34-e1da383cae48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:29.344523 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81663e-9e3d-4ae0-ab34-e1da383cae48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:29.344789 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81663e-9e3d-4ae0-ab34-e1da383cae48 admin admin] PATH_INFO: `/v3/users/b957d44949dd4272859dcb9c573d14ec` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:29.346562 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81663e-9e3d-4ae0-ab34-e1da383cae48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b957d44949dd4272859dcb9c573d14ec)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:29.348647 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81663e-9e3d-4ae0-ab34-e1da383cae48 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:29.351876 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343a852-8466-4999-82a9-c79b0c92b322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:29.352604 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343a852-8466-4999-82a9-c79b0c92b322 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:29.352881 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343a852-8466-4999-82a9-c79b0c92b322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:29.353151 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343a852-8466-4999-82a9-c79b0c92b322 admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:29.353786 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c343a852-8466-4999-82a9-c79b0c92b322 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:29.357082 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c343a852-8466-4999-82a9-c79b0c92b322 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:29.388839 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1048/2093] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:29 2022] POST /identity/v3/users => generated 293 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:30:29.392997 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343a852-8466-4999-82a9-c79b0c92b322 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:29.397422 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-c343a852-8466-4999-82a9-c79b0c92b322 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:29.412970 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-ad81663e-9e3d-4ae0-ab34-e1da383cae48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b957d44949dd4272859dcb9c573d14ec'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:29.413092 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343a852-8466-4999-82a9-c79b0c92b322 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:29.413922 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-ad81663e-9e3d-4ae0-ab34-e1da383cae48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b957d44949dd4272859dcb9c573d14ec'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:29.417793 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756e4ce-7a6e-4e16-a7a8-6bf703b4ec8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:29.418411 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756e4ce-7a6e-4e16-a7a8-6bf703b4ec8f admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:29.418734 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756e4ce-7a6e-4e16-a7a8-6bf703b4ec8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:29.419004 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756e4ce-7a6e-4e16-a7a8-6bf703b4ec8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:29.419886 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e756e4ce-7a6e-4e16-a7a8-6bf703b4ec8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:29.421491 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e756e4ce-7a6e-4e16-a7a8-6bf703b4ec8f admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:29.421860 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-ad81663e-9e3d-4ae0-ab34-e1da383cae48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b957d44949dd4272859dcb9c573d14ec'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:29.434874 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1049/2094] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:29 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:30:29.435031 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:29.438994 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756e4ce-7a6e-4e16-a7a8-6bf703b4ec8f admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:29.439806 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-ad81663e-9e3d-4ae0-ab34-e1da383cae48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b957d44949dd4272859dcb9c573d14ec'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:29.441027 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-e756e4ce-7a6e-4e16-a7a8-6bf703b4ec8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:29.455574 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756e4ce-7a6e-4e16-a7a8-6bf703b4ec8f admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:29.456147 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-ad81663e-9e3d-4ae0-ab34-e1da383cae48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b957d44949dd4272859dcb9c573d14ec'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:29.464357 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1046/2095] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:29 2022] DELETE /identity/v3/users/b957d44949dd4272859dcb9c573d14ec => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:30:29.485753 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c80dc9-77bc-4e7e-88a3-28d678a950ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:29.486443 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c80dc9-77bc-4e7e-88a3-28d678a950ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:29.486745 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c80dc9-77bc-4e7e-88a3-28d678a950ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:29.487016 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c80dc9-77bc-4e7e-88a3-28d678a950ba admin admin] PATH_INFO: `/v3/projects/2bdc6a5a22d149179b283e055bc2f576/users/a1449814a4754250852547e5dee5d09e/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:29.522845 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c80dc9-77bc-4e7e-88a3-28d678a950ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bdc6a5a22d149179b283e055bc2f576, user_id=a1449814a4754250852547e5dee5d09e, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:29.525873 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c80dc9-77bc-4e7e-88a3-28d678a950ba admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:29.563565 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1050/2096] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:29 2022] PUT /identity/v3/projects/2bdc6a5a22d149179b283e055bc2f576/users/a1449814a4754250852547e5dee5d09e/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:30:29.567556 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81663e-9e3d-4ae0-ab34-e1da383cae48 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:29.684878 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81663e-9e3d-4ae0-ab34-e1da383cae48 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:29.777554 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e623c-cbb2-4981-aa2b-7b3b6e0819a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:29.778232 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e623c-cbb2-4981-aa2b-7b3b6e0819a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:29.778472 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e623c-cbb2-4981-aa2b-7b3b6e0819a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:29.778755 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e623c-cbb2-4981-aa2b-7b3b6e0819a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:29.779442 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e623c-cbb2-4981-aa2b-7b3b6e0819a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:29.781692 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e623c-cbb2-4981-aa2b-7b3b6e0819a2 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:29.794580 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1047/2097] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:29 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:30:29.798823 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c80dc9-77bc-4e7e-88a3-28d678a950ba admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:29.831377 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c80dc9-77bc-4e7e-88a3-28d678a950ba admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:29.843180 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76f253-a226-4bfe-b16b-0a4d0931a259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:29.844298 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76f253-a226-4bfe-b16b-0a4d0931a259 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:29.845019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76f253-a226-4bfe-b16b-0a4d0931a259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:29.845395 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76f253-a226-4bfe-b16b-0a4d0931a259 admin admin] PATH_INFO: `/v3/projects/2bdc6a5a22d149179b283e055bc2f576/users/a1449814a4754250852547e5dee5d09e/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:29.847836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c76f253-a226-4bfe-b16b-0a4d0931a259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bdc6a5a22d149179b283e055bc2f576, user_id=a1449814a4754250852547e5dee5d09e, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:29.850687 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c76f253-a226-4bfe-b16b-0a4d0931a259 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:29.869251 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1051/2098] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:29 2022] PUT /identity/v3/projects/2bdc6a5a22d149179b283e055bc2f576/users/a1449814a4754250852547e5dee5d09e/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:29.914607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e623c-cbb2-4981-aa2b-7b3b6e0819a2 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:29.916700 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-266e623c-cbb2-4981-aa2b-7b3b6e0819a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:29.931012 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e623c-cbb2-4981-aa2b-7b3b6e0819a2 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:30.022983 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1073e82-a1e3-4582-8a13-cfe9a60d0145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:30.023760 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1073e82-a1e3-4582-8a13-cfe9a60d0145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:30.024057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1073e82-a1e3-4582-8a13-cfe9a60d0145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:30.024333 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1073e82-a1e3-4582-8a13-cfe9a60d0145 admin admin] PATH_INFO: `/v3/projects/826c9c6710ea443ab66a9409f2fbde77` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:30.025472 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1073e82-a1e3-4582-8a13-cfe9a60d0145 admin admin] RBAC: Authorizing `identity:delete_project(project_id=826c9c6710ea443ab66a9409f2fbde77)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:30.027655 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1073e82-a1e3-4582-8a13-cfe9a60d0145 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:30.068648 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:30.092505 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-e1073e82-a1e3-4582-8a13-cfe9a60d0145 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '826c9c6710ea443ab66a9409f2fbde77', 'request_id': 'req-e1073e82-a1e3-4582-8a13-cfe9a60d0145'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:30.102147 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-e1073e82-a1e3-4582-8a13-cfe9a60d0145 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '826c9c6710ea443ab66a9409f2fbde77', 'request_id': 'req-e1073e82-a1e3-4582-8a13-cfe9a60d0145'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:30.109525 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-e1073e82-a1e3-4582-8a13-cfe9a60d0145 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '826c9c6710ea443ab66a9409f2fbde77', 'request_id': 'req-e1073e82-a1e3-4582-8a13-cfe9a60d0145'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:30.126206 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-e1073e82-a1e3-4582-8a13-cfe9a60d0145 admin admin] The token cache is being invalidate because project 826c9c6710ea443ab66a9409f2fbde77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:30.127741 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-e1073e82-a1e3-4582-8a13-cfe9a60d0145 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:30.128524 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-e1073e82-a1e3-4582-8a13-cfe9a60d0145 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:30.130184 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1048/2099] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:29 2022] DELETE /identity/v3/projects/826c9c6710ea443ab66a9409f2fbde77 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:30.136232 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba0e98-21f1-4562-a460-74a3cf1fac77 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:30.136514 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba0e98-21f1-4562-a460-74a3cf1fac77 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:30.136784 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba0e98-21f1-4562-a460-74a3cf1fac77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:30.156579 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-40ba0e98-21f1-4562-a460-74a3cf1fac77 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:30.157794 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-40ba0e98-21f1-4562-a460-74a3cf1fac77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:30.225349 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1052/2100] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:30.235540 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5afff-c1fa-4a50-8c3b-5f7485d64df3 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:30.236129 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5afff-c1fa-4a50-8c3b-5f7485d64df3 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:30.236308 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5afff-c1fa-4a50-8c3b-5f7485d64df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:30.268825 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-0ad5afff-c1fa-4a50-8c3b-5f7485d64df3 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:30.270547 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-0ad5afff-c1fa-4a50-8c3b-5f7485d64df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:30.289133 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1049/2101] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:30.293219 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba0e98-21f1-4562-a460-74a3cf1fac77 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:30.309202 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba0e98-21f1-4562-a460-74a3cf1fac77 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:30.323418 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb7429-2033-4ea0-83c2-86e58efaa997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:30.324066 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb7429-2033-4ea0-83c2-86e58efaa997 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:30.324347 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb7429-2033-4ea0-83c2-86e58efaa997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:30.324666 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb7429-2033-4ea0-83c2-86e58efaa997 admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:30.325451 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eb7429-2033-4ea0-83c2-86e58efaa997 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:30:30.325625 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eb7429-2033-4ea0-83c2-86e58efaa997 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:30.328658 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eb7429-2033-4ea0-83c2-86e58efaa997 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:30.347667 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1053/2102] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:30:30.348530 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad5afff-c1fa-4a50-8c3b-5f7485d64df3 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:30.363915 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad5afff-c1fa-4a50-8c3b-5f7485d64df3 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:30.372008 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738ef08-ecaa-4fa0-ab53-95c1c301177c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:30.372912 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738ef08-ecaa-4fa0-ab53-95c1c301177c admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:30.373491 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738ef08-ecaa-4fa0-ab53-95c1c301177c admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:30.375086 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738ef08-ecaa-4fa0-ab53-95c1c301177c admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:30.375988 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738ef08-ecaa-4fa0-ab53-95c1c301177c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:30.378788 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738ef08-ecaa-4fa0-ab53-95c1c301177c admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:30.410853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1050/2103] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:30 2022] POST /identity/v3/projects => generated 381 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:30.414990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb7429-2033-4ea0-83c2-86e58efaa997 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:30.425934 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb7429-2033-4ea0-83c2-86e58efaa997 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:30.500100 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cfcb6c-f462-480b-80b1-9feea4eba5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:30.500892 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfcb6c-f462-480b-80b1-9feea4eba5ac admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:30.501198 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfcb6c-f462-480b-80b1-9feea4eba5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:30.501789 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfcb6c-f462-480b-80b1-9feea4eba5ac admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:30.502424 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cfcb6c-f462-480b-80b1-9feea4eba5ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:30.504415 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cfcb6c-f462-480b-80b1-9feea4eba5ac admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:30.518953 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:30.530692 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1054/2104] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:30 2022] POST /identity/v3/users => generated 291 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:30.534644 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738ef08-ecaa-4fa0-ab53-95c1c301177c admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:30.554273 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738ef08-ecaa-4fa0-ab53-95c1c301177c admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:30.560380 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af51c8-6f33-4614-a412-98d9edaa43ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:30.561009 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af51c8-6f33-4614-a412-98d9edaa43ef admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:30.561284 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af51c8-6f33-4614-a412-98d9edaa43ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:30.561554 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af51c8-6f33-4614-a412-98d9edaa43ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:30.562456 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af51c8-6f33-4614-a412-98d9edaa43ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:30.564239 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af51c8-6f33-4614-a412-98d9edaa43ef admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:30.575689 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1051/2105] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:30.581705 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cfcb6c-f462-480b-80b1-9feea4eba5ac admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:30.612163 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cfcb6c-f462-480b-80b1-9feea4eba5ac admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:30.618083 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4448b4-428e-43a4-bb4d-983767ef7eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:30.618753 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4448b4-428e-43a4-bb4d-983767ef7eff admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:30.619034 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4448b4-428e-43a4-bb4d-983767ef7eff admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:30.619451 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4448b4-428e-43a4-bb4d-983767ef7eff admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:30.620106 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4448b4-428e-43a4-bb4d-983767ef7eff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:30.621814 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4448b4-428e-43a4-bb4d-983767ef7eff admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:30.635494 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1055/2106] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:30:30.640255 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af51c8-6f33-4614-a412-98d9edaa43ef admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:30.653450 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af51c8-6f33-4614-a412-98d9edaa43ef admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:30.659374 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c57515-9263-4e8e-a46f-275d8d01945c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:30.659926 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c57515-9263-4e8e-a46f-275d8d01945c admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:30.660230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c57515-9263-4e8e-a46f-275d8d01945c admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:30.660462 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c57515-9263-4e8e-a46f-275d8d01945c admin admin] PATH_INFO: `/v3/projects/6db3ee893c6f4d56977866fa38c684d0/users/d28b6e27d2524870928c21419cda2daa/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:30.681297 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c57515-9263-4e8e-a46f-275d8d01945c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6db3ee893c6f4d56977866fa38c684d0, user_id=d28b6e27d2524870928c21419cda2daa, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:30.684337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c57515-9263-4e8e-a46f-275d8d01945c admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:30.701202 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1052/2107] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:30 2022] PUT /identity/v3/projects/6db3ee893c6f4d56977866fa38c684d0/users/d28b6e27d2524870928c21419cda2daa/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:30:30.713141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd38edf-3d1b-458c-b48f-44cbd8d12654 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:30.713557 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd38edf-3d1b-458c-b48f-44cbd8d12654 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:30.713924 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd38edf-3d1b-458c-b48f-44cbd8d12654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:30.765492 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-7cd38edf-3d1b-458c-b48f-44cbd8d12654 None None] MFA Rules not processed for user `d28b6e27d2524870928c21419cda2daa`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:30.767275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-7cd38edf-3d1b-458c-b48f-44cbd8d12654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:30.883071 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1056/2108] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:30.889262 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c57515-9263-4e8e-a46f-275d8d01945c admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:31.008008 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c57515-9263-4e8e-a46f-275d8d01945c admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:31.013120 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10038db-aac2-4823-97d2-234950c70c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:31.013795 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10038db-aac2-4823-97d2-234950c70c1f admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:31.014625 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10038db-aac2-4823-97d2-234950c70c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:31.014945 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10038db-aac2-4823-97d2-234950c70c1f admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:31.015685 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10038db-aac2-4823-97d2-234950c70c1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:31.018907 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10038db-aac2-4823-97d2-234950c70c1f admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:31.073421 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1053/2109] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:30 2022] POST /identity/v3/projects => generated 375 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:31.077603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd38edf-3d1b-458c-b48f-44cbd8d12654 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:31.102953 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd38edf-3d1b-458c-b48f-44cbd8d12654 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:31.196279 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf5f9ac-0579-4039-a303-cbdd3e507935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:31.197112 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf5f9ac-0579-4039-a303-cbdd3e507935 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:31.197492 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf5f9ac-0579-4039-a303-cbdd3e507935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:31.197862 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf5f9ac-0579-4039-a303-cbdd3e507935 admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:31.198626 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf5f9ac-0579-4039-a303-cbdd3e507935 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:31.200753 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf5f9ac-0579-4039-a303-cbdd3e507935 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:31.250725 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1057/2110] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:31 2022] POST /identity/v3/users => generated 294 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:31.252182 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10038db-aac2-4823-97d2-234950c70c1f admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:31.271915 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10038db-aac2-4823-97d2-234950c70c1f admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:31.280723 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d958e9-179e-43b4-9a49-0e8fb1c62cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:31.281414 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d958e9-179e-43b4-9a49-0e8fb1c62cb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:31.281833 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d958e9-179e-43b4-9a49-0e8fb1c62cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:31.282152 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d958e9-179e-43b4-9a49-0e8fb1c62cb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:31.283038 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d958e9-179e-43b4-9a49-0e8fb1c62cb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:31.285220 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d958e9-179e-43b4-9a49-0e8fb1c62cb5 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:31.301126 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1054/2111] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:31.305254 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf5f9ac-0579-4039-a303-cbdd3e507935 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:31.324155 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf5f9ac-0579-4039-a303-cbdd3e507935 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:31.331168 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f743837-86a1-452e-b5a4-b607781f6f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:31.331973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f743837-86a1-452e-b5a4-b607781f6f0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:31.332318 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f743837-86a1-452e-b5a4-b607781f6f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:31.333470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f743837-86a1-452e-b5a4-b607781f6f0d admin admin] PATH_INFO: `/v3/projects/5bd0ca4aa4654234bf93d1e99e589bc3/users/760b88a09d9943a0bb5e924603af91c5/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:31.358766 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f743837-86a1-452e-b5a4-b607781f6f0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bd0ca4aa4654234bf93d1e99e589bc3, user_id=760b88a09d9943a0bb5e924603af91c5, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:31.361506 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f743837-86a1-452e-b5a4-b607781f6f0d admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:31.381622 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1058/2112] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:31 2022] PUT /identity/v3/projects/5bd0ca4aa4654234bf93d1e99e589bc3/users/760b88a09d9943a0bb5e924603af91c5/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:31.385662 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d958e9-179e-43b4-9a49-0e8fb1c62cb5 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:31.397783 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d958e9-179e-43b4-9a49-0e8fb1c62cb5 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:31.456958 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:31.550006 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45eef45-f559-48d6-af40-8674400695a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:31.552149 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45eef45-f559-48d6-af40-8674400695a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:31.552895 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45eef45-f559-48d6-af40-8674400695a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:31.557708 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45eef45-f559-48d6-af40-8674400695a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:31.559355 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45eef45-f559-48d6-af40-8674400695a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:31.561065 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45eef45-f559-48d6-af40-8674400695a8 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:31.580866 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1055/2113] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:31.586712 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f743837-86a1-452e-b5a4-b607781f6f0d admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:31.603504 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f743837-86a1-452e-b5a4-b607781f6f0d admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:31.616204 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47376e0d-8ada-4f11-a42c-21601d70a463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:31.616874 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47376e0d-8ada-4f11-a42c-21601d70a463 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:31.617168 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47376e0d-8ada-4f11-a42c-21601d70a463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:31.617450 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47376e0d-8ada-4f11-a42c-21601d70a463 admin admin] PATH_INFO: `/v3/projects/5bd0ca4aa4654234bf93d1e99e589bc3/users/760b88a09d9943a0bb5e924603af91c5/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:31.621376 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47376e0d-8ada-4f11-a42c-21601d70a463 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bd0ca4aa4654234bf93d1e99e589bc3, user_id=760b88a09d9943a0bb5e924603af91c5, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:31.624153 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47376e0d-8ada-4f11-a42c-21601d70a463 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:31.661409 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1059/2114] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:31 2022] PUT /identity/v3/projects/5bd0ca4aa4654234bf93d1e99e589bc3/users/760b88a09d9943a0bb5e924603af91c5/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:31.663373 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2c219-3107-4a33-84df-4e507dde9386 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:31.663664 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2c219-3107-4a33-84df-4e507dde9386 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:31.663942 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2c219-3107-4a33-84df-4e507dde9386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:31.718291 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-2bb2c219-3107-4a33-84df-4e507dde9386 None None] MFA Rules not processed for user `760b88a09d9943a0bb5e924603af91c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:31.719866 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-2bb2c219-3107-4a33-84df-4e507dde9386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:31.852451 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1056/2115] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:31.864429 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47376e0d-8ada-4f11-a42c-21601d70a463 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:31.866286 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-47376e0d-8ada-4f11-a42c-21601d70a463 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:31.915109 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:31.984684 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47376e0d-8ada-4f11-a42c-21601d70a463 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:31.989821 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68dae86-b0a8-40d3-9ff2-c0bd40e627af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:31.990618 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68dae86-b0a8-40d3-9ff2-c0bd40e627af service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:31.991035 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68dae86-b0a8-40d3-9ff2-c0bd40e627af service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:31.991335 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68dae86-b0a8-40d3-9ff2-c0bd40e627af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:32.007437 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68dae86-b0a8-40d3-9ff2-c0bd40e627af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:32.008932 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68dae86-b0a8-40d3-9ff2-c0bd40e627af service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:32.021062 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1060/2116] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:31 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:30:32.631127 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2c219-3107-4a33-84df-4e507dde9386 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:32.726971 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2c219-3107-4a33-84df-4e507dde9386 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:32.742412 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5c1b8-dc42-456a-bbc9-c331a7cb76aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:32.743534 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5c1b8-dc42-456a-bbc9-c331a7cb76aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:32.743534 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5c1b8-dc42-456a-bbc9-c331a7cb76aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:32.743844 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5c1b8-dc42-456a-bbc9-c331a7cb76aa admin admin] PATH_INFO: `/v3/users/d28b6e27d2524870928c21419cda2daa` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:32.748006 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5c1b8-dc42-456a-bbc9-c331a7cb76aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=d28b6e27d2524870928c21419cda2daa)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:32.751847 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5c1b8-dc42-456a-bbc9-c331a7cb76aa admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:32.885337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-59c5c1b8-dc42-456a-bbc9-c331a7cb76aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd28b6e27d2524870928c21419cda2daa'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:32.886030 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-59c5c1b8-dc42-456a-bbc9-c331a7cb76aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd28b6e27d2524870928c21419cda2daa'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:32.906374 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:32.908942 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-59c5c1b8-dc42-456a-bbc9-c331a7cb76aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd28b6e27d2524870928c21419cda2daa'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:32.936209 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-59c5c1b8-dc42-456a-bbc9-c331a7cb76aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd28b6e27d2524870928c21419cda2daa'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:32.964546 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-59c5c1b8-dc42-456a-bbc9-c331a7cb76aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd28b6e27d2524870928c21419cda2daa'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:32.972642 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1057/2117] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:32 2022] DELETE /identity/v3/users/d28b6e27d2524870928c21419cda2daa => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:32.977160 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac5655-8b35-46eb-933f-072c9704fa54 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:32.977862 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac5655-8b35-46eb-933f-072c9704fa54 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:32.977862 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac5655-8b35-46eb-933f-072c9704fa54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:33.019532 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-a8ac5655-8b35-46eb-933f-072c9704fa54 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:33.020890 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-a8ac5655-8b35-46eb-933f-072c9704fa54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:33.206090 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1061/2118] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:30:33.233984 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5c1b8-dc42-456a-bbc9-c331a7cb76aa admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:33.236744 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-59c5c1b8-dc42-456a-bbc9-c331a7cb76aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:33.481071 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5c1b8-dc42-456a-bbc9-c331a7cb76aa admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:33.490877 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af95804-81ed-4528-83f8-ab33cfecff51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:33.491540 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af95804-81ed-4528-83f8-ab33cfecff51 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:33.491826 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af95804-81ed-4528-83f8-ab33cfecff51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:33.496904 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af95804-81ed-4528-83f8-ab33cfecff51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:33.541965 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af95804-81ed-4528-83f8-ab33cfecff51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:33.546491 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af95804-81ed-4528-83f8-ab33cfecff51 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:33.585827 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1058/2119] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:35.189597 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac5655-8b35-46eb-933f-072c9704fa54 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:35.192157 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-a8ac5655-8b35-46eb-933f-072c9704fa54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:35.215810 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac5655-8b35-46eb-933f-072c9704fa54 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:35.224859 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb56ea-e873-471f-aafd-99b3bf8a7780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:35.226105 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb56ea-e873-471f-aafd-99b3bf8a7780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:35.226625 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb56ea-e873-471f-aafd-99b3bf8a7780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:35.231838 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb56ea-e873-471f-aafd-99b3bf8a7780 admin admin] PATH_INFO: `/v3/projects/6db3ee893c6f4d56977866fa38c684d0` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:35.237260 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb56ea-e873-471f-aafd-99b3bf8a7780 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6db3ee893c6f4d56977866fa38c684d0)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:35.241015 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb56ea-e873-471f-aafd-99b3bf8a7780 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:35.366404 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-74fb56ea-e873-471f-aafd-99b3bf8a7780 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6db3ee893c6f4d56977866fa38c684d0', 'request_id': 'req-74fb56ea-e873-471f-aafd-99b3bf8a7780'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:35.383767 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-74fb56ea-e873-471f-aafd-99b3bf8a7780 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6db3ee893c6f4d56977866fa38c684d0', 'request_id': 'req-74fb56ea-e873-471f-aafd-99b3bf8a7780'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:35.401367 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-74fb56ea-e873-471f-aafd-99b3bf8a7780 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6db3ee893c6f4d56977866fa38c684d0', 'request_id': 'req-74fb56ea-e873-471f-aafd-99b3bf8a7780'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:35.423783 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-74fb56ea-e873-471f-aafd-99b3bf8a7780 admin admin] The token cache is being invalidate because project 6db3ee893c6f4d56977866fa38c684d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:35.424133 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-74fb56ea-e873-471f-aafd-99b3bf8a7780 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:35.425398 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-74fb56ea-e873-471f-aafd-99b3bf8a7780 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:35.426909 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1062/2120] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:35 2022] DELETE /identity/v3/projects/6db3ee893c6f4d56977866fa38c684d0 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:35.432102 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af95804-81ed-4528-83f8-ab33cfecff51 service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:35.445475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af95804-81ed-4528-83f8-ab33cfecff51 service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:35.622168 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f881cb-1619-42fd-a422-b69b162f9852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:35.622845 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f881cb-1619-42fd-a422-b69b162f9852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:35.623118 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f881cb-1619-42fd-a422-b69b162f9852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:35.623384 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f881cb-1619-42fd-a422-b69b162f9852 admin admin] PATH_INFO: `/v3/users/760b88a09d9943a0bb5e924603af91c5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:35.630586 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f881cb-1619-42fd-a422-b69b162f9852 admin admin] RBAC: Authorizing `identity:delete_user(user_id=760b88a09d9943a0bb5e924603af91c5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:35.634777 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f881cb-1619-42fd-a422-b69b162f9852 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:35.830646 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-64f881cb-1619-42fd-a422-b69b162f9852 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '760b88a09d9943a0bb5e924603af91c5'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:35.834682 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-64f881cb-1619-42fd-a422-b69b162f9852 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '760b88a09d9943a0bb5e924603af91c5'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:35.860292 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-64f881cb-1619-42fd-a422-b69b162f9852 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '760b88a09d9943a0bb5e924603af91c5'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:35.906290 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-64f881cb-1619-42fd-a422-b69b162f9852 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '760b88a09d9943a0bb5e924603af91c5'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:35.913985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:35.935458 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-64f881cb-1619-42fd-a422-b69b162f9852 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '760b88a09d9943a0bb5e924603af91c5'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:35.949273 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1059/2121] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:35 2022] DELETE /identity/v3/users/760b88a09d9943a0bb5e924603af91c5 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:36.679828 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb56ea-e873-471f-aafd-99b3bf8a7780 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:36.683936 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-74fb56ea-e873-471f-aafd-99b3bf8a7780 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:36.742593 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb56ea-e873-471f-aafd-99b3bf8a7780 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:36.896233 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:36.911232 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092df99-6db9-4418-a310-767452cb9f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:36.912164 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092df99-6db9-4418-a310-767452cb9f62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:36.912621 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092df99-6db9-4418-a310-767452cb9f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:36.919398 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092df99-6db9-4418-a310-767452cb9f62 admin admin] PATH_INFO: `/v3/projects/5bd0ca4aa4654234bf93d1e99e589bc3` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:36.926677 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092df99-6db9-4418-a310-767452cb9f62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bd0ca4aa4654234bf93d1e99e589bc3)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:36.934460 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092df99-6db9-4418-a310-767452cb9f62 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:37.079197 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-2092df99-6db9-4418-a310-767452cb9f62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bd0ca4aa4654234bf93d1e99e589bc3', 'request_id': 'req-2092df99-6db9-4418-a310-767452cb9f62'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:37.096352 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-2092df99-6db9-4418-a310-767452cb9f62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bd0ca4aa4654234bf93d1e99e589bc3', 'request_id': 'req-2092df99-6db9-4418-a310-767452cb9f62'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:37.106252 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-2092df99-6db9-4418-a310-767452cb9f62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bd0ca4aa4654234bf93d1e99e589bc3', 'request_id': 'req-2092df99-6db9-4418-a310-767452cb9f62'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:37.125310 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-2092df99-6db9-4418-a310-767452cb9f62 admin admin] The token cache is being invalidate because project 5bd0ca4aa4654234bf93d1e99e589bc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:37.125906 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-2092df99-6db9-4418-a310-767452cb9f62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:37.128298 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-2092df99-6db9-4418-a310-767452cb9f62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:37.130897 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1063/2122] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:36 2022] DELETE /identity/v3/projects/5bd0ca4aa4654234bf93d1e99e589bc3 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:37.135496 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a70954-e0ec-4983-84ef-4fb2f22335aa None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:37.136540 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a70954-e0ec-4983-84ef-4fb2f22335aa None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:37.137009 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a70954-e0ec-4983-84ef-4fb2f22335aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:37.202983 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-d1a70954-e0ec-4983-84ef-4fb2f22335aa None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:37.205236 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-d1a70954-e0ec-4983-84ef-4fb2f22335aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:37.465341 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1060/2123] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:37.473701 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587436d6-d295-4e3c-94bd-770211837556 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:37.474209 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587436d6-d295-4e3c-94bd-770211837556 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:37.474489 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587436d6-d295-4e3c-94bd-770211837556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:37.518830 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-587436d6-d295-4e3c-94bd-770211837556 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:37.524778 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-587436d6-d295-4e3c-94bd-770211837556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:37.553534 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a70954-e0ec-4983-84ef-4fb2f22335aa None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:37.559436 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1064/2124] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:37.566959 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a70954-e0ec-4983-84ef-4fb2f22335aa None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:37.572354 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a535a67-57c1-426c-bae2-fb0b407cf907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:37.573066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a535a67-57c1-426c-bae2-fb0b407cf907 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:37.575371 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a535a67-57c1-426c-bae2-fb0b407cf907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:37.575743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a535a67-57c1-426c-bae2-fb0b407cf907 admin admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:37.576485 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a535a67-57c1-426c-bae2-fb0b407cf907 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:30:37.576859 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a535a67-57c1-426c-bae2-fb0b407cf907 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:37.579579 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a535a67-57c1-426c-bae2-fb0b407cf907 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:37.621870 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1061/2125] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:30:37.629272 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587436d6-d295-4e3c-94bd-770211837556 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:37.641695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587436d6-d295-4e3c-94bd-770211837556 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:37.651753 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3610d4a-baa4-444b-b95b-88bfe3bfd122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:37.652493 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3610d4a-baa4-444b-b95b-88bfe3bfd122 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:37.652828 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3610d4a-baa4-444b-b95b-88bfe3bfd122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:37.653127 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3610d4a-baa4-444b-b95b-88bfe3bfd122 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:37.653884 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3610d4a-baa4-444b-b95b-88bfe3bfd122 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:37.656226 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3610d4a-baa4-444b-b95b-88bfe3bfd122 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:37.730708 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1065/2126] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:37 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:37.732200 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a535a67-57c1-426c-bae2-fb0b407cf907 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:37.744417 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a535a67-57c1-426c-bae2-fb0b407cf907 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:37.938150 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907eb64e-316b-46e8-ac59-af228c7b931d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:37.939455 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907eb64e-316b-46e8-ac59-af228c7b931d admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:37.939743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907eb64e-316b-46e8-ac59-af228c7b931d admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:37.940013 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907eb64e-316b-46e8-ac59-af228c7b931d admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:37.940673 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907eb64e-316b-46e8-ac59-af228c7b931d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:37.946332 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907eb64e-316b-46e8-ac59-af228c7b931d admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:37.990627 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1062/2127] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:37 2022] POST /identity/v3/users => generated 288 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:37.997440 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3610d4a-baa4-444b-b95b-88bfe3bfd122 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:38.035925 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3610d4a-baa4-444b-b95b-88bfe3bfd122 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:38.048058 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4e3ee-f570-49a7-87dc-e3fb79752720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:38.049480 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4e3ee-f570-49a7-87dc-e3fb79752720 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:38.051637 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4e3ee-f570-49a7-87dc-e3fb79752720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:38.051637 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4e3ee-f570-49a7-87dc-e3fb79752720 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:38.054465 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d4e3ee-f570-49a7-87dc-e3fb79752720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:38.057056 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d4e3ee-f570-49a7-87dc-e3fb79752720 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:38.116078 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1066/2128] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:38.118770 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907eb64e-316b-46e8-ac59-af228c7b931d admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:38.130219 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907eb64e-316b-46e8-ac59-af228c7b931d admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:38.148376 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fd7e4f-50c9-4537-9614-d97ea1cf4df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:38.149848 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd7e4f-50c9-4537-9614-d97ea1cf4df4 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:38.150389 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd7e4f-50c9-4537-9614-d97ea1cf4df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:38.150863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd7e4f-50c9-4537-9614-d97ea1cf4df4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:38.152434 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fd7e4f-50c9-4537-9614-d97ea1cf4df4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:38.173256 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fd7e4f-50c9-4537-9614-d97ea1cf4df4 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:38.201653 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1063/2129] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:38.208017 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4e3ee-f570-49a7-87dc-e3fb79752720 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:38.225050 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4e3ee-f570-49a7-87dc-e3fb79752720 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:38.238906 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54757e28-4980-4870-8dfa-ef97f275930b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:38.240157 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54757e28-4980-4870-8dfa-ef97f275930b admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:38.240942 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54757e28-4980-4870-8dfa-ef97f275930b admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:38.241662 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54757e28-4980-4870-8dfa-ef97f275930b admin admin] PATH_INFO: `/v3/projects/7b40715a099c492ba0a81159932f24d1/users/2e95f74de85543ff87383198258bf72d/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:38.275580 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54757e28-4980-4870-8dfa-ef97f275930b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b40715a099c492ba0a81159932f24d1, user_id=2e95f74de85543ff87383198258bf72d, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:38.279044 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54757e28-4980-4870-8dfa-ef97f275930b admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:38.306461 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1067/2130] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:38 2022] PUT /identity/v3/projects/7b40715a099c492ba0a81159932f24d1/users/2e95f74de85543ff87383198258bf72d/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:38.317804 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f3040-f112-485b-a56f-6fd8a40efca6 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:38.318955 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f3040-f112-485b-a56f-6fd8a40efca6 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:38.319267 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f3040-f112-485b-a56f-6fd8a40efca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:38.420109 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-1b6f3040-f112-485b-a56f-6fd8a40efca6 None None] MFA Rules not processed for user `2e95f74de85543ff87383198258bf72d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:38.421369 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-1b6f3040-f112-485b-a56f-6fd8a40efca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:38.585643 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:38.620010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39e848-d6ae-4608-a50b-02543157d037 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:38.620191 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39e848-d6ae-4608-a50b-02543157d037 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:38.620422 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39e848-d6ae-4608-a50b-02543157d037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:38.629509 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1064/2131] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:38.635370 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6f3040-f112-485b-a56f-6fd8a40efca6 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:38.679820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-8a39e848-d6ae-4608-a50b-02543157d037 None None] MFA Rules not processed for user `a1449814a4754250852547e5dee5d09e`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:38.681140 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-8a39e848-d6ae-4608-a50b-02543157d037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:38.789906 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6f3040-f112-485b-a56f-6fd8a40efca6 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:38.805093 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace79fc3-9d30-4e4b-a1da-9a03204f13d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:38.805812 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace79fc3-9d30-4e4b-a1da-9a03204f13d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:38.806083 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace79fc3-9d30-4e4b-a1da-9a03204f13d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:38.806352 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace79fc3-9d30-4e4b-a1da-9a03204f13d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:38.807237 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace79fc3-9d30-4e4b-a1da-9a03204f13d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:38.811039 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace79fc3-9d30-4e4b-a1da-9a03204f13d4 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:38.872951 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1068/2132] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:38.883137 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1065/2133] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:38 2022] POST /identity/v3/projects => generated 375 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:38.887780 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39e848-d6ae-4608-a50b-02543157d037 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:38.895174 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:38.899088 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace79fc3-9d30-4e4b-a1da-9a03204f13d4 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:38.900406 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39e848-d6ae-4608-a50b-02543157d037 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:38.901076 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-ace79fc3-9d30-4e4b-a1da-9a03204f13d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:38.983809 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e95aa-f4cc-4902-8719-f941b4ec3d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:38.984760 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e95aa-f4cc-4902-8719-f941b4ec3d40 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:38.985078 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e95aa-f4cc-4902-8719-f941b4ec3d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:38.985352 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e95aa-f4cc-4902-8719-f941b4ec3d40 admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:38.985986 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212e95aa-f4cc-4902-8719-f941b4ec3d40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:38.988227 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212e95aa-f4cc-4902-8719-f941b4ec3d40 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:39.022428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1069/2134] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:38 2022] POST /identity/v3/users => generated 294 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:39.031969 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e95aa-f4cc-4902-8719-f941b4ec3d40 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:39.044795 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e95aa-f4cc-4902-8719-f941b4ec3d40 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:39.052445 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace79fc3-9d30-4e4b-a1da-9a03204f13d4 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:39.053403 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd820855-e12c-43a6-9508-121eb13f3ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:39.054045 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd820855-e12c-43a6-9508-121eb13f3ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:39.054313 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd820855-e12c-43a6-9508-121eb13f3ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:39.054581 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd820855-e12c-43a6-9508-121eb13f3ef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:39.055451 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd820855-e12c-43a6-9508-121eb13f3ef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:39.057059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd820855-e12c-43a6-9508-121eb13f3ef4 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:39.061088 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c3847f-51e6-44f0-9867-9453d286d36c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:39.061814 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3847f-51e6-44f0-9867-9453d286d36c service nova] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:39.062148 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3847f-51e6-44f0-9867-9453d286d36c service nova] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:39.062482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3847f-51e6-44f0-9867-9453d286d36c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:39.064707 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-b7c3847f-51e6-44f0-9867-9453d286d36c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:39.072677 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1070/2135] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:39.076933 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd820855-e12c-43a6-9508-121eb13f3ef4 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:39.092431 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd820855-e12c-43a6-9508-121eb13f3ef4 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:39.101524 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4285b-1957-4496-876a-19051db830d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:39.102140 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4285b-1957-4496-876a-19051db830d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:39.102416 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4285b-1957-4496-876a-19051db830d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:39.102794 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4285b-1957-4496-876a-19051db830d1 admin admin] PATH_INFO: `/v3/projects/ddb8ef40fca341ac9ee6cab525bce9f5/users/c29af1dce83d4ca4928aa9269e4d8f63/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:39.133571 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d4285b-1957-4496-876a-19051db830d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddb8ef40fca341ac9ee6cab525bce9f5, user_id=c29af1dce83d4ca4928aa9269e4d8f63, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:39.136647 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d4285b-1957-4496-876a-19051db830d1 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:39.157615 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c3847f-51e6-44f0-9867-9453d286d36c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:39.162240 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c3847f-51e6-44f0-9867-9453d286d36c service nova] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:39.173359 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1071/2136] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:39 2022] PUT /identity/v3/projects/ddb8ef40fca341ac9ee6cab525bce9f5/users/c29af1dce83d4ca4928aa9269e4d8f63/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:39.175036 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4285b-1957-4496-876a-19051db830d1 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:39.179176 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1066/2137] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:30:39.358635 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4285b-1957-4496-876a-19051db830d1 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:39.371379 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862e807-48e3-44d3-8e46-c88f93187a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:39.372630 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862e807-48e3-44d3-8e46-c88f93187a60 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:39.374170 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862e807-48e3-44d3-8e46-c88f93187a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:39.374170 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862e807-48e3-44d3-8e46-c88f93187a60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:39.374170 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f862e807-48e3-44d3-8e46-c88f93187a60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:39.377314 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f862e807-48e3-44d3-8e46-c88f93187a60 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:39.412081 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1072/2138] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:39.416482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c3847f-51e6-44f0-9867-9453d286d36c service nova] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:39.441281 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c3847f-51e6-44f0-9867-9453d286d36c service nova] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:39.452412 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc5027-1ae8-4b31-892c-d924039f9816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:39.457199 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc5027-1ae8-4b31-892c-d924039f9816 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:39.457499 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc5027-1ae8-4b31-892c-d924039f9816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:39.457783 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc5027-1ae8-4b31-892c-d924039f9816 admin admin] PATH_INFO: `/v3/projects/ddb8ef40fca341ac9ee6cab525bce9f5/users/c29af1dce83d4ca4928aa9269e4d8f63/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:39.461734 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc5027-1ae8-4b31-892c-d924039f9816 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddb8ef40fca341ac9ee6cab525bce9f5, user_id=c29af1dce83d4ca4928aa9269e4d8f63, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:39.464685 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc5027-1ae8-4b31-892c-d924039f9816 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:39.481900 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:39.499341 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1067/2139] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:39 2022] PUT /identity/v3/projects/ddb8ef40fca341ac9ee6cab525bce9f5/users/c29af1dce83d4ca4928aa9269e4d8f63/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:39.501278 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261980da-aa16-4bf2-9f1b-6e63909d6424 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:39.502288 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261980da-aa16-4bf2-9f1b-6e63909d6424 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:39.502908 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261980da-aa16-4bf2-9f1b-6e63909d6424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:39.545248 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-261980da-aa16-4bf2-9f1b-6e63909d6424 None None] MFA Rules not processed for user `c29af1dce83d4ca4928aa9269e4d8f63`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:39.546569 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-261980da-aa16-4bf2-9f1b-6e63909d6424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:39.694520 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:39.818505 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1073/2140] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:39 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:39.832802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc5027-1ae8-4b31-892c-d924039f9816 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:39.836598 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-28bc5027-1ae8-4b31-892c-d924039f9816 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:40.065091 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc5027-1ae8-4b31-892c-d924039f9816 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:40.074258 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafff2c4-dd5d-44ad-af26-dd6c5b246321 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:40.075984 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafff2c4-dd5d-44ad-af26-dd6c5b246321 service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:40.075984 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafff2c4-dd5d-44ad-af26-dd6c5b246321 service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:40.075984 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafff2c4-dd5d-44ad-af26-dd6c5b246321 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:40.254646 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafff2c4-dd5d-44ad-af26-dd6c5b246321 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:40.257054 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafff2c4-dd5d-44ad-af26-dd6c5b246321 service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:40.305031 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1068/2141] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:41.931462 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af079ee-6aad-47e9-8961-db839bb96023 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:41.932893 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af079ee-6aad-47e9-8961-db839bb96023 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:41.933365 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af079ee-6aad-47e9-8961-db839bb96023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:42.011136 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-3af079ee-6aad-47e9-8961-db839bb96023 None None] MFA Rules not processed for user `a47b81144eba4283aac610c600e73ed3`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:42.017945 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-3af079ee-6aad-47e9-8961-db839bb96023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:42.099176 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579eb5cd-e0c5-469c-a383-a22bdedc8ee0 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:42.099778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579eb5cd-e0c5-469c-a383-a22bdedc8ee0 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:42.099778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579eb5cd-e0c5-469c-a383-a22bdedc8ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:42.141730 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1074/2142] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:42.163081 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-579eb5cd-e0c5-469c-a383-a22bdedc8ee0 None None] MFA Rules not processed for user `3f9fe0442c2b41fa822780b1b7b3b807`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:42.164311 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-579eb5cd-e0c5-469c-a383-a22bdedc8ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:42.262376 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1069/2143] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:43.863489 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af079ee-6aad-47e9-8961-db839bb96023 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:44.074013 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af079ee-6aad-47e9-8961-db839bb96023 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:44.084333 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebaa777-6363-460f-ad99-6f23fc7c08ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:44.086210 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebaa777-6363-460f-ad99-6f23fc7c08ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:44.087228 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebaa777-6363-460f-ad99-6f23fc7c08ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:44.087703 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebaa777-6363-460f-ad99-6f23fc7c08ea admin admin] PATH_INFO: `/v3/users/2e95f74de85543ff87383198258bf72d` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:44.097721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebaa777-6363-460f-ad99-6f23fc7c08ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e95f74de85543ff87383198258bf72d)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:44.102353 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebaa777-6363-460f-ad99-6f23fc7c08ea admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:44.258916 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-2ebaa777-6363-460f-ad99-6f23fc7c08ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e95f74de85543ff87383198258bf72d'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:44.268122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-2ebaa777-6363-460f-ad99-6f23fc7c08ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e95f74de85543ff87383198258bf72d'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:44.269164 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-2ebaa777-6363-460f-ad99-6f23fc7c08ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e95f74de85543ff87383198258bf72d'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:44.301712 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-2ebaa777-6363-460f-ad99-6f23fc7c08ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e95f74de85543ff87383198258bf72d'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:44.324392 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-2ebaa777-6363-460f-ad99-6f23fc7c08ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e95f74de85543ff87383198258bf72d'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:44.345890 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1075/2144] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:43 2022] DELETE /identity/v3/users/2e95f74de85543ff87383198258bf72d => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:44.346948 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0c8f8-ea21-4031-af91-e3d2abf24c61 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:44.347718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0c8f8-ea21-4031-af91-e3d2abf24c61 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:44.348138 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0c8f8-ea21-4031-af91-e3d2abf24c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:44.390362 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-31d0c8f8-ea21-4031-af91-e3d2abf24c61 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:44.396919 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-31d0c8f8-ea21-4031-af91-e3d2abf24c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:44.516941 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:44.546422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1070/2145] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:44.552476 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebaa777-6363-460f-ad99-6f23fc7c08ea admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:44.553935 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-2ebaa777-6363-460f-ad99-6f23fc7c08ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:44.565937 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:44.801127 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebaa777-6363-460f-ad99-6f23fc7c08ea admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:44.830213 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ecbe51-ceab-4edc-a852-417a284aaf56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:44.830213 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecbe51-ceab-4edc-a852-417a284aaf56 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:44.830213 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecbe51-ceab-4edc-a852-417a284aaf56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:44.831017 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecbe51-ceab-4edc-a852-417a284aaf56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:44.851834 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ecbe51-ceab-4edc-a852-417a284aaf56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:44.856301 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ecbe51-ceab-4edc-a852-417a284aaf56 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:44.904495 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1076/2146] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:45.536092 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0c8f8-ea21-4031-af91-e3d2abf24c61 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:45.538189 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-31d0c8f8-ea21-4031-af91-e3d2abf24c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:45.552777 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0c8f8-ea21-4031-af91-e3d2abf24c61 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:45.561850 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d0982e-b21b-4cf7-bea3-8bd313a93157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:45.563239 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0982e-b21b-4cf7-bea3-8bd313a93157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:45.563727 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0982e-b21b-4cf7-bea3-8bd313a93157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:45.564340 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0982e-b21b-4cf7-bea3-8bd313a93157 admin admin] PATH_INFO: `/v3/projects/7b40715a099c492ba0a81159932f24d1` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:45.565799 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d0982e-b21b-4cf7-bea3-8bd313a93157 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b40715a099c492ba0a81159932f24d1)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:45.568274 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d0982e-b21b-4cf7-bea3-8bd313a93157 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:45.653193 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a4d0982e-b21b-4cf7-bea3-8bd313a93157 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b40715a099c492ba0a81159932f24d1', 'request_id': 'req-a4d0982e-b21b-4cf7-bea3-8bd313a93157'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:45.661097 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a4d0982e-b21b-4cf7-bea3-8bd313a93157 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b40715a099c492ba0a81159932f24d1', 'request_id': 'req-a4d0982e-b21b-4cf7-bea3-8bd313a93157'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:45.667670 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a4d0982e-b21b-4cf7-bea3-8bd313a93157 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b40715a099c492ba0a81159932f24d1', 'request_id': 'req-a4d0982e-b21b-4cf7-bea3-8bd313a93157'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:45.687628 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a4d0982e-b21b-4cf7-bea3-8bd313a93157 admin admin] The token cache is being invalidate because project 7b40715a099c492ba0a81159932f24d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:45.687922 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a4d0982e-b21b-4cf7-bea3-8bd313a93157 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:45.688804 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a4d0982e-b21b-4cf7-bea3-8bd313a93157 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:45.690884 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1071/2147] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:45 2022] DELETE /identity/v3/projects/7b40715a099c492ba0a81159932f24d1 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:45.694947 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ecbe51-ceab-4edc-a852-417a284aaf56 service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:45.706329 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ecbe51-ceab-4edc-a852-417a284aaf56 service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:45.779579 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e622d9-1c41-4041-b81e-19697ec0a3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:45.780136 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e622d9-1c41-4041-b81e-19697ec0a3bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:45.780414 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e622d9-1c41-4041-b81e-19697ec0a3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:45.780681 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e622d9-1c41-4041-b81e-19697ec0a3bc admin admin] PATH_INFO: `/v3/users/c29af1dce83d4ca4928aa9269e4d8f63` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:45.782248 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e622d9-1c41-4041-b81e-19697ec0a3bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c29af1dce83d4ca4928aa9269e4d8f63)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:45.784028 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e622d9-1c41-4041-b81e-19697ec0a3bc admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:45.849842 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-46e622d9-1c41-4041-b81e-19697ec0a3bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c29af1dce83d4ca4928aa9269e4d8f63'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:45.857981 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-46e622d9-1c41-4041-b81e-19697ec0a3bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c29af1dce83d4ca4928aa9269e4d8f63'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:45.859361 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-46e622d9-1c41-4041-b81e-19697ec0a3bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c29af1dce83d4ca4928aa9269e4d8f63'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:45.878988 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-46e622d9-1c41-4041-b81e-19697ec0a3bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c29af1dce83d4ca4928aa9269e4d8f63'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:45.888992 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-46e622d9-1c41-4041-b81e-19697ec0a3bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c29af1dce83d4ca4928aa9269e4d8f63'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:45.898936 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1077/2148] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:30:45 2022] DELETE /identity/v3/users/c29af1dce83d4ca4928aa9269e4d8f63 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:46.250093 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d0982e-b21b-4cf7-bea3-8bd313a93157 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:46.252886 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-a4d0982e-b21b-4cf7-bea3-8bd313a93157 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:46.270743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d0982e-b21b-4cf7-bea3-8bd313a93157 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:46.397339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a721bbf6-58da-47d4-a38e-1b47b0a421f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:46.398081 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721bbf6-58da-47d4-a38e-1b47b0a421f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:46.398428 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721bbf6-58da-47d4-a38e-1b47b0a421f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:46.398802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721bbf6-58da-47d4-a38e-1b47b0a421f7 admin admin] PATH_INFO: `/v3/projects/ddb8ef40fca341ac9ee6cab525bce9f5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:46.400349 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a721bbf6-58da-47d4-a38e-1b47b0a421f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddb8ef40fca341ac9ee6cab525bce9f5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:46.402985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a721bbf6-58da-47d4-a38e-1b47b0a421f7 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:46.471457 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a721bbf6-58da-47d4-a38e-1b47b0a421f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddb8ef40fca341ac9ee6cab525bce9f5', 'request_id': 'req-a721bbf6-58da-47d4-a38e-1b47b0a421f7'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:46.482801 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a721bbf6-58da-47d4-a38e-1b47b0a421f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddb8ef40fca341ac9ee6cab525bce9f5', 'request_id': 'req-a721bbf6-58da-47d4-a38e-1b47b0a421f7'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:46.489927 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a721bbf6-58da-47d4-a38e-1b47b0a421f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddb8ef40fca341ac9ee6cab525bce9f5', 'request_id': 'req-a721bbf6-58da-47d4-a38e-1b47b0a421f7'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:46.508782 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a721bbf6-58da-47d4-a38e-1b47b0a421f7 admin admin] The token cache is being invalidate because project ddb8ef40fca341ac9ee6cab525bce9f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:30:46.509182 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a721bbf6-58da-47d4-a38e-1b47b0a421f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:46.510792 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a721bbf6-58da-47d4-a38e-1b47b0a421f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:30:46.514829 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1072/2149] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:30:46 2022] DELETE /identity/v3/projects/ddb8ef40fca341ac9ee6cab525bce9f5 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:46.519597 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f660cf-1496-49f2-84eb-4e4bd2955888 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:46.520620 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f660cf-1496-49f2-84eb-4e4bd2955888 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:46.521044 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f660cf-1496-49f2-84eb-4e4bd2955888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:46.548451 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-c7f660cf-1496-49f2-84eb-4e4bd2955888 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:46.549657 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-c7f660cf-1496-49f2-84eb-4e4bd2955888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:46.574073 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:46.640537 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1078/2150] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:46.650651 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b07ec-4736-4bf0-b95f-4a8ae0ce61dd None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:46.651805 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b07ec-4736-4bf0-b95f-4a8ae0ce61dd None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:46.652109 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b07ec-4736-4bf0-b95f-4a8ae0ce61dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:46.675015 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-9b0b07ec-4736-4bf0-b95f-4a8ae0ce61dd None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:46.676408 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-9b0b07ec-4736-4bf0-b95f-4a8ae0ce61dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:46.693653 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1073/2151] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:30:46.698263 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f660cf-1496-49f2-84eb-4e4bd2955888 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:46.711727 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f660cf-1496-49f2-84eb-4e4bd2955888 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:46.717874 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902a0c0-358c-400e-a3f1-161678ddb550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:46.718587 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902a0c0-358c-400e-a3f1-161678ddb550 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:46.718954 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902a0c0-358c-400e-a3f1-161678ddb550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:46.719221 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902a0c0-358c-400e-a3f1-161678ddb550 admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:46.720660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8902a0c0-358c-400e-a3f1-161678ddb550 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:30:46.721001 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8902a0c0-358c-400e-a3f1-161678ddb550 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:46.724105 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8902a0c0-358c-400e-a3f1-161678ddb550 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:46.749382 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1079/2152] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:30:46.755849 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b07ec-4736-4bf0-b95f-4a8ae0ce61dd None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:46.769215 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:46.773852 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b07ec-4736-4bf0-b95f-4a8ae0ce61dd None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:46.780685 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706faf57-a81b-450d-8c5d-fd51493d89e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:46.781263 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706faf57-a81b-450d-8c5d-fd51493d89e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:46.781522 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706faf57-a81b-450d-8c5d-fd51493d89e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:46.782034 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706faf57-a81b-450d-8c5d-fd51493d89e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:46.783245 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706faf57-a81b-450d-8c5d-fd51493d89e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:46.787055 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706faf57-a81b-450d-8c5d-fd51493d89e2 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:46.823408 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1074/2153] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:46 2022] POST /identity/v3/projects => generated 383 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:46.828308 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902a0c0-358c-400e-a3f1-161678ddb550 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:46.842311 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902a0c0-358c-400e-a3f1-161678ddb550 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:46.907000 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45055144-e4fc-425c-9522-cae4505046f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:46.907708 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45055144-e4fc-425c-9522-cae4505046f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:46.907982 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45055144-e4fc-425c-9522-cae4505046f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:46.908253 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45055144-e4fc-425c-9522-cae4505046f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:46.908961 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45055144-e4fc-425c-9522-cae4505046f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:46.910992 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45055144-e4fc-425c-9522-cae4505046f7 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:46.935759 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1080/2154] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:46 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:30:46.939972 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706faf57-a81b-450d-8c5d-fd51493d89e2 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:46.952044 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706faf57-a81b-450d-8c5d-fd51493d89e2 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:46.957830 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad15b8-e4cf-4df2-83c8-19996d888dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:46.958443 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad15b8-e4cf-4df2-83c8-19996d888dbf admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:46.958746 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad15b8-e4cf-4df2-83c8-19996d888dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:46.959024 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad15b8-e4cf-4df2-83c8-19996d888dbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:46.959782 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad15b8-e4cf-4df2-83c8-19996d888dbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:46.961444 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad15b8-e4cf-4df2-83c8-19996d888dbf admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:46.974570 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1075/2155] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:46.980012 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45055144-e4fc-425c-9522-cae4505046f7 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:46.990968 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45055144-e4fc-425c-9522-cae4505046f7 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:46.996277 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07745b6-c0a3-4b10-9277-bf4dd954a294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:46.996997 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07745b6-c0a3-4b10-9277-bf4dd954a294 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:46.997344 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07745b6-c0a3-4b10-9277-bf4dd954a294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:46.997680 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07745b6-c0a3-4b10-9277-bf4dd954a294 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:46.998426 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07745b6-c0a3-4b10-9277-bf4dd954a294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:47.000082 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07745b6-c0a3-4b10-9277-bf4dd954a294 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:47.012171 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1081/2156] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:47.016477 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad15b8-e4cf-4df2-83c8-19996d888dbf admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:47.027424 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad15b8-e4cf-4df2-83c8-19996d888dbf admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:47.032699 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f538504-65ce-41b9-be76-8ce7d6c690fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:47.033508 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f538504-65ce-41b9-be76-8ce7d6c690fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:47.033681 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f538504-65ce-41b9-be76-8ce7d6c690fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:47.033947 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f538504-65ce-41b9-be76-8ce7d6c690fd admin admin] PATH_INFO: `/v3/projects/18485bf83e07404eaa99cf1ec498baf1/users/d536f7e140a446c6b32c1eeff282b622/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:47.049982 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f538504-65ce-41b9-be76-8ce7d6c690fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=18485bf83e07404eaa99cf1ec498baf1, user_id=d536f7e140a446c6b32c1eeff282b622, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:47.052677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f538504-65ce-41b9-be76-8ce7d6c690fd admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:47.067003 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1076/2157] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:47 2022] PUT /identity/v3/projects/18485bf83e07404eaa99cf1ec498baf1/users/d536f7e140a446c6b32c1eeff282b622/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:47.077446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d65368-4819-4bfa-8744-833c5a7c6f3b None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:47.077686 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d65368-4819-4bfa-8744-833c5a7c6f3b None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:47.077919 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d65368-4819-4bfa-8744-833c5a7c6f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:47.109362 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-80d65368-4819-4bfa-8744-833c5a7c6f3b None None] MFA Rules not processed for user `d536f7e140a446c6b32c1eeff282b622`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:47.110736 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-80d65368-4819-4bfa-8744-833c5a7c6f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:47.174546 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1082/2158] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:47.179388 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f538504-65ce-41b9-be76-8ce7d6c690fd admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:47.245870 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f538504-65ce-41b9-be76-8ce7d6c690fd admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:47.251027 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a5cd9b-e35b-40c4-a0aa-881363c5a23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:47.252001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5cd9b-e35b-40c4-a0aa-881363c5a23d admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:47.252283 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5cd9b-e35b-40c4-a0aa-881363c5a23d admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:47.252555 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5cd9b-e35b-40c4-a0aa-881363c5a23d admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:47.253211 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a5cd9b-e35b-40c4-a0aa-881363c5a23d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:47.255192 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a5cd9b-e35b-40c4-a0aa-881363c5a23d admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:47.285443 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1077/2159] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:47 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:47.290135 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d65368-4819-4bfa-8744-833c5a7c6f3b None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:47.300849 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d65368-4819-4bfa-8744-833c5a7c6f3b None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:47.321702 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:47.368322 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb0627-50d4-4e5e-8c52-bfdabe59b76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:47.369024 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb0627-50d4-4e5e-8c52-bfdabe59b76d admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:47.369307 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb0627-50d4-4e5e-8c52-bfdabe59b76d admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:47.369577 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb0627-50d4-4e5e-8c52-bfdabe59b76d admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:47.370230 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb0627-50d4-4e5e-8c52-bfdabe59b76d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:47.372185 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb0627-50d4-4e5e-8c52-bfdabe59b76d admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:47.399907 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1083/2160] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:47 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:30:47.404118 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a5cd9b-e35b-40c4-a0aa-881363c5a23d admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:47.414903 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a5cd9b-e35b-40c4-a0aa-881363c5a23d admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:47.419640 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d4f50-5e4d-4d13-808e-250859a44a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:47.420273 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d4f50-5e4d-4d13-808e-250859a44a0c admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:47.420551 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d4f50-5e4d-4d13-808e-250859a44a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:47.420819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d4f50-5e4d-4d13-808e-250859a44a0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:47.421503 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372d4f50-5e4d-4d13-808e-250859a44a0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:47.423293 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372d4f50-5e4d-4d13-808e-250859a44a0c admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:47.434740 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1078/2161] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:47.439054 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb0627-50d4-4e5e-8c52-bfdabe59b76d admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:47.449017 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb0627-50d4-4e5e-8c52-bfdabe59b76d admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:47.453601 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d110a-d6f1-4605-ab59-e44579822480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:47.454323 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d110a-d6f1-4605-ab59-e44579822480 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:47.454598 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d110a-d6f1-4605-ab59-e44579822480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:47.454902 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d110a-d6f1-4605-ab59-e44579822480 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:47.455545 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d110a-d6f1-4605-ab59-e44579822480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:47.457152 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d110a-d6f1-4605-ab59-e44579822480 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:47.467444 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1084/2162] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:47.471738 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d4f50-5e4d-4d13-808e-250859a44a0c admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:47.481780 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d4f50-5e4d-4d13-808e-250859a44a0c admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:47.491435 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b8623c-9107-40eb-b1cc-27198557a8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:47.492135 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8623c-9107-40eb-b1cc-27198557a8dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:47.492488 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8623c-9107-40eb-b1cc-27198557a8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:47.492762 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8623c-9107-40eb-b1cc-27198557a8dd admin admin] PATH_INFO: `/v3/projects/46945c142b434a2ca8dcc65c9b82429c/users/6d58c1383304420fa3448cd61b4f6157/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:47.511970 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b8623c-9107-40eb-b1cc-27198557a8dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=46945c142b434a2ca8dcc65c9b82429c, user_id=6d58c1383304420fa3448cd61b4f6157, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:47.514615 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b8623c-9107-40eb-b1cc-27198557a8dd admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:47.534714 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1079/2163] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:47 2022] PUT /identity/v3/projects/46945c142b434a2ca8dcc65c9b82429c/users/6d58c1383304420fa3448cd61b4f6157/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:30:47.611946 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f112b95-db27-44b4-988a-3af8df789260 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:47.612393 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f112b95-db27-44b4-988a-3af8df789260 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:47.612509 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f112b95-db27-44b4-988a-3af8df789260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:47.667873 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-6f112b95-db27-44b4-988a-3af8df789260 None None] MFA Rules not processed for user `6d58c1383304420fa3448cd61b4f6157`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:47.670194 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-6f112b95-db27-44b4-988a-3af8df789260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:47.744796 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1085/2164] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:47.749488 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b8623c-9107-40eb-b1cc-27198557a8dd admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:47.820005 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:47.825335 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b8623c-9107-40eb-b1cc-27198557a8dd admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:47.831181 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206004f4-2a7a-442c-990e-3499cf58bfa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:47.831796 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206004f4-2a7a-442c-990e-3499cf58bfa3 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:47.832614 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206004f4-2a7a-442c-990e-3499cf58bfa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:47.832614 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206004f4-2a7a-442c-990e-3499cf58bfa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:47.832923 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206004f4-2a7a-442c-990e-3499cf58bfa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:47.834888 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206004f4-2a7a-442c-990e-3499cf58bfa3 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:47.869456 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1080/2165] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:30:47 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:30:47.870888 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f112b95-db27-44b4-988a-3af8df789260 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:47.883917 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f112b95-db27-44b4-988a-3af8df789260 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:47.951550 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b965f-5e5d-4933-aca2-a9a96f2a36a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:47.952167 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b965f-5e5d-4933-aca2-a9a96f2a36a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:47.952443 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b965f-5e5d-4933-aca2-a9a96f2a36a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:47.952715 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b965f-5e5d-4933-aca2-a9a96f2a36a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:47.953330 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b965f-5e5d-4933-aca2-a9a96f2a36a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:47.955332 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b965f-5e5d-4933-aca2-a9a96f2a36a7 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:47.981505 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1086/2166] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:30:47 2022] POST /identity/v3/users => generated 299 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:30:47.982768 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206004f4-2a7a-442c-990e-3499cf58bfa3 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:47.993775 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206004f4-2a7a-442c-990e-3499cf58bfa3 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:47.998595 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c147fe-c2e0-42a6-ae37-89bd39c45fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:47.999264 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c147fe-c2e0-42a6-ae37-89bd39c45fbe admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:47.999593 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c147fe-c2e0-42a6-ae37-89bd39c45fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:47.999873 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c147fe-c2e0-42a6-ae37-89bd39c45fbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:48.000547 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c147fe-c2e0-42a6-ae37-89bd39c45fbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:48.002919 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c147fe-c2e0-42a6-ae37-89bd39c45fbe admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:48.014495 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1081/2167] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:30:48.019309 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b965f-5e5d-4933-aca2-a9a96f2a36a7 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:48.030297 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b965f-5e5d-4933-aca2-a9a96f2a36a7 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:48.036301 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc499e-aabf-4fe7-a71d-6085097d961d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:48.037010 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc499e-aabf-4fe7-a71d-6085097d961d admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:48.037269 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc499e-aabf-4fe7-a71d-6085097d961d admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:48.037534 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc499e-aabf-4fe7-a71d-6085097d961d admin admin] PATH_INFO: `/v3/projects/08f287e1b9084b668b8862d28a31c268/users/59752dba564d4b2ea92e65f7faa96596/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:48.054712 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc499e-aabf-4fe7-a71d-6085097d961d admin admin] RBAC: Authorizing `identity:create_grant(project_id=08f287e1b9084b668b8862d28a31c268, user_id=59752dba564d4b2ea92e65f7faa96596, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:48.057487 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc499e-aabf-4fe7-a71d-6085097d961d admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:48.069366 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:48.082486 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1087/2168] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:48 2022] PUT /identity/v3/projects/08f287e1b9084b668b8862d28a31c268/users/59752dba564d4b2ea92e65f7faa96596/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:48.086576 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c147fe-c2e0-42a6-ae37-89bd39c45fbe admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:48.096687 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c147fe-c2e0-42a6-ae37-89bd39c45fbe admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:48.160768 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa4fb4-a9cb-4e4d-90b1-88c3691edf06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:48.161403 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa4fb4-a9cb-4e4d-90b1-88c3691edf06 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:48.161681 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa4fb4-a9cb-4e4d-90b1-88c3691edf06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:48.161957 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa4fb4-a9cb-4e4d-90b1-88c3691edf06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:48.162632 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aa4fb4-a9cb-4e4d-90b1-88c3691edf06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:48.164294 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aa4fb4-a9cb-4e4d-90b1-88c3691edf06 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:48.177046 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1082/2169] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:30:48.180958 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc499e-aabf-4fe7-a71d-6085097d961d admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:48.192832 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc499e-aabf-4fe7-a71d-6085097d961d admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:48.198768 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1effaa-7c85-4b10-afb8-bea674ef7434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:48.199526 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1effaa-7c85-4b10-afb8-bea674ef7434 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:48.199812 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1effaa-7c85-4b10-afb8-bea674ef7434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:48.200076 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1effaa-7c85-4b10-afb8-bea674ef7434 admin admin] PATH_INFO: `/v3/projects/08f287e1b9084b668b8862d28a31c268/users/59752dba564d4b2ea92e65f7faa96596/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:48.202246 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1effaa-7c85-4b10-afb8-bea674ef7434 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08f287e1b9084b668b8862d28a31c268, user_id=59752dba564d4b2ea92e65f7faa96596, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:48.205176 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1effaa-7c85-4b10-afb8-bea674ef7434 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:48.223869 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1088/2170] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:30:48 2022] PUT /identity/v3/projects/08f287e1b9084b668b8862d28a31c268/users/59752dba564d4b2ea92e65f7faa96596/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:30:48.234355 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec946de-cbd5-44a9-a8ec-b38b274c1250 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:48.234645 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec946de-cbd5-44a9-a8ec-b38b274c1250 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:48.234945 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec946de-cbd5-44a9-a8ec-b38b274c1250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:48.269181 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-3ec946de-cbd5-44a9-a8ec-b38b274c1250 None None] MFA Rules not processed for user `59752dba564d4b2ea92e65f7faa96596`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:30:48.270445 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-3ec946de-cbd5-44a9-a8ec-b38b274c1250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:48.348931 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1083/2171] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:30:48 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:30:48.362540 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1effaa-7c85-4b10-afb8-bea674ef7434 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:48.365251 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-ed1effaa-7c85-4b10-afb8-bea674ef7434 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:30:48.446433 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1effaa-7c85-4b10-afb8-bea674ef7434 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:48.450366 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e602a89-a17d-45ac-9f90-8312110e3067 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:48.451040 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e602a89-a17d-45ac-9f90-8312110e3067 service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:48.451318 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e602a89-a17d-45ac-9f90-8312110e3067 service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:48.451596 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e602a89-a17d-45ac-9f90-8312110e3067 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:48.516486 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e602a89-a17d-45ac-9f90-8312110e3067 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:48.518110 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e602a89-a17d-45ac-9f90-8312110e3067 service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:48.526882 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1089/2172] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:57.616947 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec946de-cbd5-44a9-a8ec-b38b274c1250 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:57.630544 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec946de-cbd5-44a9-a8ec-b38b274c1250 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:57.642467 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cdbf07-3a03-4aa3-8f8b-9550306a3d38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:57.642467 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cdbf07-3a03-4aa3-8f8b-9550306a3d38 service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:57.642915 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cdbf07-3a03-4aa3-8f8b-9550306a3d38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:57.642999 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cdbf07-3a03-4aa3-8f8b-9550306a3d38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:57.782310 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:57.799905 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cdbf07-3a03-4aa3-8f8b-9550306a3d38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:57.806161 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cdbf07-3a03-4aa3-8f8b-9550306a3d38 service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:57.830541 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1084/2173] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:30:58.523147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=55,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:30:58.686650 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e602a89-a17d-45ac-9f90-8312110e3067 service cinder] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:30:58.698900 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e602a89-a17d-45ac-9f90-8312110e3067 service cinder] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:30:58.706312 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d53db7-89e5-4e6a-91dd-12bc4866fb4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:30:58.706892 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d53db7-89e5-4e6a-91dd-12bc4866fb4d service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:30:58.707167 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d53db7-89e5-4e6a-91dd-12bc4866fb4d service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:30:58.707431 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d53db7-89e5-4e6a-91dd-12bc4866fb4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:30:58.730403 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d53db7-89e5-4e6a-91dd-12bc4866fb4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:30:58.731927 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d53db7-89e5-4e6a-91dd-12bc4866fb4d service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:30:58.742875 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1090/2174] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:30:58 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:31:00.048886 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cdbf07-3a03-4aa3-8f8b-9550306a3d38 service cinder] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:00.050726 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-28cdbf07-3a03-4aa3-8f8b-9550306a3d38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:00.160027 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cdbf07-3a03-4aa3-8f8b-9550306a3d38 service cinder] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:00.165565 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c79e50-accc-4eb7-9e76-0d68d066a8a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:00.166186 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c79e50-accc-4eb7-9e76-0d68d066a8a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:00.166454 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c79e50-accc-4eb7-9e76-0d68d066a8a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:00.166742 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c79e50-accc-4eb7-9e76-0d68d066a8a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:00.168595 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-35c79e50-accc-4eb7-9e76-0d68d066a8a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:00.301694 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c79e50-accc-4eb7-9e76-0d68d066a8a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:00.303468 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c79e50-accc-4eb7-9e76-0d68d066a8a6 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:00.332514 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1085/2175] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:00 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:31:01.645312 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89221cd4-8649-4ec2-9961-ba4302b1bd76 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:01.646056 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89221cd4-8649-4ec2-9961-ba4302b1bd76 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:01.646056 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89221cd4-8649-4ec2-9961-ba4302b1bd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:01.689679 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-89221cd4-8649-4ec2-9961-ba4302b1bd76 None None] MFA Rules not processed for user `e4f9bb9b60a246f3a84a4d4618c3f527`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:01.691907 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-89221cd4-8649-4ec2-9961-ba4302b1bd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:01.790359 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1091/2176] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:31:01 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:31:02.715751 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c79e50-accc-4eb7-9e76-0d68d066a8a6 service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:02.717517 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-35c79e50-accc-4eb7-9e76-0d68d066a8a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:02.801454 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c79e50-accc-4eb7-9e76-0d68d066a8a6 service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:02.806839 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb402420-4f02-4382-95e8-2e3da034c73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:02.808016 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb402420-4f02-4382-95e8-2e3da034c73f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:02.808445 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb402420-4f02-4382-95e8-2e3da034c73f admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:02.808845 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb402420-4f02-4382-95e8-2e3da034c73f admin admin] PATH_INFO: `/v3/users/fbfdde83696d486eb39384d1bb01bf8e` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:02.810240 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb402420-4f02-4382-95e8-2e3da034c73f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbfdde83696d486eb39384d1bb01bf8e)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:02.812589 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb402420-4f02-4382-95e8-2e3da034c73f admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:02.875587 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:02.881591 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-eb402420-4f02-4382-95e8-2e3da034c73f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbfdde83696d486eb39384d1bb01bf8e'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:02.882893 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-eb402420-4f02-4382-95e8-2e3da034c73f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbfdde83696d486eb39384d1bb01bf8e'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:02.892582 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-eb402420-4f02-4382-95e8-2e3da034c73f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbfdde83696d486eb39384d1bb01bf8e'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:02.905862 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-eb402420-4f02-4382-95e8-2e3da034c73f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbfdde83696d486eb39384d1bb01bf8e'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:02.916991 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-eb402420-4f02-4382-95e8-2e3da034c73f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbfdde83696d486eb39384d1bb01bf8e'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:02.928740 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdf047-729d-4e85-bafe-7a1815b20232 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:02.929317 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdf047-729d-4e85-bafe-7a1815b20232 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:02.929317 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdf047-729d-4e85-bafe-7a1815b20232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:02.932374 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1086/2177] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:31:02 2022] DELETE /identity/v3/users/fbfdde83696d486eb39384d1bb01bf8e => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:02.950569 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-4afdf047-729d-4e85-bafe-7a1815b20232 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:02.951903 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-4afdf047-729d-4e85-bafe-7a1815b20232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:03.024240 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1092/2178] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:31:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:03.137838 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb402420-4f02-4382-95e8-2e3da034c73f admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:03.139591 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-eb402420-4f02-4382-95e8-2e3da034c73f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:03.158778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb402420-4f02-4382-95e8-2e3da034c73f admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:03.164331 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086158c4-54dc-45cf-aa38-5af0a5181628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:03.164977 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086158c4-54dc-45cf-aa38-5af0a5181628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:03.165251 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086158c4-54dc-45cf-aa38-5af0a5181628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:03.165517 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086158c4-54dc-45cf-aa38-5af0a5181628 admin admin] PATH_INFO: `/v3/users/d536f7e140a446c6b32c1eeff282b622` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:03.166680 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086158c4-54dc-45cf-aa38-5af0a5181628 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d536f7e140a446c6b32c1eeff282b622)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:03.168642 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086158c4-54dc-45cf-aa38-5af0a5181628 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:03.235831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-086158c4-54dc-45cf-aa38-5af0a5181628 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd536f7e140a446c6b32c1eeff282b622'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:03.237181 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-086158c4-54dc-45cf-aa38-5af0a5181628 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd536f7e140a446c6b32c1eeff282b622'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:03.247171 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-086158c4-54dc-45cf-aa38-5af0a5181628 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd536f7e140a446c6b32c1eeff282b622'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:03.265493 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-086158c4-54dc-45cf-aa38-5af0a5181628 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd536f7e140a446c6b32c1eeff282b622'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:03.277939 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-086158c4-54dc-45cf-aa38-5af0a5181628 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd536f7e140a446c6b32c1eeff282b622'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:03.286961 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1087/2179] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:31:03 2022] DELETE /identity/v3/users/d536f7e140a446c6b32c1eeff282b622 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:03.291189 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b824e2f-b3de-452c-96a2-fafdb1d90be4 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:03.291628 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b824e2f-b3de-452c-96a2-fafdb1d90be4 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:03.292040 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b824e2f-b3de-452c-96a2-fafdb1d90be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:03.300828 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086158c4-54dc-45cf-aa38-5af0a5181628 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:03.303296 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-086158c4-54dc-45cf-aa38-5af0a5181628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:03.321678 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-0b824e2f-b3de-452c-96a2-fafdb1d90be4 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:03.323665 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-0b824e2f-b3de-452c-96a2-fafdb1d90be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:03.414500 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1093/2180] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:31:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:03.447960 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b824e2f-b3de-452c-96a2-fafdb1d90be4 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:03.449684 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-0b824e2f-b3de-452c-96a2-fafdb1d90be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:03.458348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086158c4-54dc-45cf-aa38-5af0a5181628 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:03.463345 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656fb46-ffd6-45bf-8e7e-8a42ead4703d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:03.473860 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b824e2f-b3de-452c-96a2-fafdb1d90be4 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:03.474559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656fb46-ffd6-45bf-8e7e-8a42ead4703d service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:03.475107 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656fb46-ffd6-45bf-8e7e-8a42ead4703d service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:03.475513 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656fb46-ffd6-45bf-8e7e-8a42ead4703d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:03.479648 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-c656fb46-ffd6-45bf-8e7e-8a42ead4703d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:03.487490 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a4e7d-2b6b-4359-bb34-89135bf5a78d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:03.488219 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a4e7d-2b6b-4359-bb34-89135bf5a78d service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:03.488496 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a4e7d-2b6b-4359-bb34-89135bf5a78d service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:03.489282 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a4e7d-2b6b-4359-bb34-89135bf5a78d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:03.515024 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c656fb46-ffd6-45bf-8e7e-8a42ead4703d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:03.516662 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c656fb46-ffd6-45bf-8e7e-8a42ead4703d service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:03.520056 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a4e7d-2b6b-4359-bb34-89135bf5a78d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:03.522170 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a4e7d-2b6b-4359-bb34-89135bf5a78d service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:03.543882 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1088/2181] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:31:03.556208 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1094/2182] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:31:04.724547 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656fb46-ffd6-45bf-8e7e-8a42ead4703d service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:04.726090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-c656fb46-ffd6-45bf-8e7e-8a42ead4703d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:04.738772 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656fb46-ffd6-45bf-8e7e-8a42ead4703d service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:04.744403 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b8c135-f63f-45fd-959c-8a2cd2dbcfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:04.745282 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8c135-f63f-45fd-959c-8a2cd2dbcfe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:04.745756 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8c135-f63f-45fd-959c-8a2cd2dbcfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:04.746085 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8c135-f63f-45fd-959c-8a2cd2dbcfe3 admin admin] PATH_INFO: `/v3/projects/e9426908dddc440cb2092287386250e9` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:04.747332 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b8c135-f63f-45fd-959c-8a2cd2dbcfe3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9426908dddc440cb2092287386250e9)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:04.749611 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b8c135-f63f-45fd-959c-8a2cd2dbcfe3 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:04.761493 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a4e7d-2b6b-4359-bb34-89135bf5a78d service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:04.762932 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-325a4e7d-2b6b-4359-bb34-89135bf5a78d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:04.781966 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a4e7d-2b6b-4359-bb34-89135bf5a78d service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:04.790201 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc982d94-33de-4ec4-8e9d-40ec22a815b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:04.791357 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc982d94-33de-4ec4-8e9d-40ec22a815b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:04.791642 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc982d94-33de-4ec4-8e9d-40ec22a815b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:04.791917 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc982d94-33de-4ec4-8e9d-40ec22a815b5 admin admin] PATH_INFO: `/v3/projects/18485bf83e07404eaa99cf1ec498baf1` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:04.814048 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc982d94-33de-4ec4-8e9d-40ec22a815b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18485bf83e07404eaa99cf1ec498baf1)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:04.820143 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc982d94-33de-4ec4-8e9d-40ec22a815b5 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:04.830905 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:04.844276 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:04.854735 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-51b8c135-f63f-45fd-959c-8a2cd2dbcfe3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9426908dddc440cb2092287386250e9', 'request_id': 'req-51b8c135-f63f-45fd-959c-8a2cd2dbcfe3'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:04.864228 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-51b8c135-f63f-45fd-959c-8a2cd2dbcfe3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9426908dddc440cb2092287386250e9', 'request_id': 'req-51b8c135-f63f-45fd-959c-8a2cd2dbcfe3'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:04.875672 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-51b8c135-f63f-45fd-959c-8a2cd2dbcfe3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9426908dddc440cb2092287386250e9', 'request_id': 'req-51b8c135-f63f-45fd-959c-8a2cd2dbcfe3'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:04.897022 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-51b8c135-f63f-45fd-959c-8a2cd2dbcfe3 admin admin] The token cache is being invalidate because project e9426908dddc440cb2092287386250e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:31:04.897454 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-51b8c135-f63f-45fd-959c-8a2cd2dbcfe3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:04.898330 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-51b8c135-f63f-45fd-959c-8a2cd2dbcfe3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:04.903416 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-dc982d94-33de-4ec4-8e9d-40ec22a815b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18485bf83e07404eaa99cf1ec498baf1', 'request_id': 'req-dc982d94-33de-4ec4-8e9d-40ec22a815b5'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:04.904992 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1089/2183] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:31:04 2022] DELETE /identity/v3/projects/e9426908dddc440cb2092287386250e9 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:04.937989 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-dc982d94-33de-4ec4-8e9d-40ec22a815b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18485bf83e07404eaa99cf1ec498baf1', 'request_id': 'req-dc982d94-33de-4ec4-8e9d-40ec22a815b5'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:04.949109 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-dc982d94-33de-4ec4-8e9d-40ec22a815b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18485bf83e07404eaa99cf1ec498baf1', 'request_id': 'req-dc982d94-33de-4ec4-8e9d-40ec22a815b5'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:04.967704 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-dc982d94-33de-4ec4-8e9d-40ec22a815b5 admin admin] The token cache is being invalidate because project 18485bf83e07404eaa99cf1ec498baf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:31:04.968314 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-dc982d94-33de-4ec4-8e9d-40ec22a815b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:04.969384 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-dc982d94-33de-4ec4-8e9d-40ec22a815b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:04.971602 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1095/2184] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:31:04 2022] DELETE /identity/v3/projects/18485bf83e07404eaa99cf1ec498baf1 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:04.975637 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b8c135-f63f-45fd-959c-8a2cd2dbcfe3 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:04.977672 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-51b8c135-f63f-45fd-959c-8a2cd2dbcfe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:05.142601 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b8c135-f63f-45fd-959c-8a2cd2dbcfe3 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:05.230029 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1709667c-be64-4462-ad07-f388d7519e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:05.230354 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1709667c-be64-4462-ad07-f388d7519e45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:05.230457 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1709667c-be64-4462-ad07-f388d7519e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:05.230633 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1709667c-be64-4462-ad07-f388d7519e45 admin admin] PATH_INFO: `/v3/users/6d58c1383304420fa3448cd61b4f6157` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:05.232058 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1709667c-be64-4462-ad07-f388d7519e45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d58c1383304420fa3448cd61b4f6157)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:05.234583 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1709667c-be64-4462-ad07-f388d7519e45 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:05.344066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-1709667c-be64-4462-ad07-f388d7519e45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d58c1383304420fa3448cd61b4f6157'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:05.344750 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-1709667c-be64-4462-ad07-f388d7519e45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d58c1383304420fa3448cd61b4f6157'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:05.356590 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-1709667c-be64-4462-ad07-f388d7519e45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d58c1383304420fa3448cd61b4f6157'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:05.370610 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:05.377977 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-1709667c-be64-4462-ad07-f388d7519e45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d58c1383304420fa3448cd61b4f6157'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:05.397986 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-1709667c-be64-4462-ad07-f388d7519e45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d58c1383304420fa3448cd61b4f6157'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:05.407566 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1090/2185] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:31:04 2022] DELETE /identity/v3/users/6d58c1383304420fa3448cd61b4f6157 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:05.976618 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc982d94-33de-4ec4-8e9d-40ec22a815b5 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:05.978085 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-dc982d94-33de-4ec4-8e9d-40ec22a815b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:06.008694 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc982d94-33de-4ec4-8e9d-40ec22a815b5 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:06.088194 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6099d-bb30-4b57-8a59-0e7acba4e88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:06.088963 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6099d-bb30-4b57-8a59-0e7acba4e88a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:06.089331 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6099d-bb30-4b57-8a59-0e7acba4e88a admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:06.089667 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6099d-bb30-4b57-8a59-0e7acba4e88a admin admin] PATH_INFO: `/v3/projects/46945c142b434a2ca8dcc65c9b82429c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:06.090916 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6099d-bb30-4b57-8a59-0e7acba4e88a admin admin] RBAC: Authorizing `identity:delete_project(project_id=46945c142b434a2ca8dcc65c9b82429c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:06.093097 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6099d-bb30-4b57-8a59-0e7acba4e88a admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:06.191140 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-83e6099d-bb30-4b57-8a59-0e7acba4e88a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46945c142b434a2ca8dcc65c9b82429c', 'request_id': 'req-83e6099d-bb30-4b57-8a59-0e7acba4e88a'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:06.215214 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-83e6099d-bb30-4b57-8a59-0e7acba4e88a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46945c142b434a2ca8dcc65c9b82429c', 'request_id': 'req-83e6099d-bb30-4b57-8a59-0e7acba4e88a'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:06.228451 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-83e6099d-bb30-4b57-8a59-0e7acba4e88a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46945c142b434a2ca8dcc65c9b82429c', 'request_id': 'req-83e6099d-bb30-4b57-8a59-0e7acba4e88a'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:06.248969 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-83e6099d-bb30-4b57-8a59-0e7acba4e88a admin admin] The token cache is being invalidate because project 46945c142b434a2ca8dcc65c9b82429c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:31:06.249449 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-83e6099d-bb30-4b57-8a59-0e7acba4e88a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:06.250470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-83e6099d-bb30-4b57-8a59-0e7acba4e88a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:06.252746 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1096/2186] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:31:05 2022] DELETE /identity/v3/projects/46945c142b434a2ca8dcc65c9b82429c => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:06.257098 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1709667c-be64-4462-ad07-f388d7519e45 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:06.258610 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-1709667c-be64-4462-ad07-f388d7519e45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:06.297997 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1709667c-be64-4462-ad07-f388d7519e45 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:06.405509 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51aa72-7430-4686-ae46-4aac07511bbf None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:06.405973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51aa72-7430-4686-ae46-4aac07511bbf None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:06.406339 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51aa72-7430-4686-ae46-4aac07511bbf None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:06.407999 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1097/2187] 10.222.0.30 () {54 vars in 961 bytes} [Wed Nov 2 07:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:31:06.413492 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0559f-669a-4499-9057-9c4c667c3b85 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:06.414325 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0559f-669a-4499-9057-9c4c667c3b85 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:06.414325 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0559f-669a-4499-9057-9c4c667c3b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:06.434064 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10b9aa-059f-42ee-8ba2-44ae5312b492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:06.434963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10b9aa-059f-42ee-8ba2-44ae5312b492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:06.435341 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10b9aa-059f-42ee-8ba2-44ae5312b492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:06.435712 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10b9aa-059f-42ee-8ba2-44ae5312b492 admin admin] PATH_INFO: `/v3/users/59752dba564d4b2ea92e65f7faa96596` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:06.437266 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd10b9aa-059f-42ee-8ba2-44ae5312b492 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59752dba564d4b2ea92e65f7faa96596)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:06.440353 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd10b9aa-059f-42ee-8ba2-44ae5312b492 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:06.483444 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-36a0559f-669a-4499-9057-9c4c667c3b85 None None] MFA Rules not processed for user `932d3076fb4b426d8c83d02eb60d8063`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:06.484793 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-36a0559f-669a-4499-9057-9c4c667c3b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:06.522984 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:06.578940 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-bd10b9aa-059f-42ee-8ba2-44ae5312b492 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59752dba564d4b2ea92e65f7faa96596'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:06.579643 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-bd10b9aa-059f-42ee-8ba2-44ae5312b492 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59752dba564d4b2ea92e65f7faa96596'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:06.592400 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-bd10b9aa-059f-42ee-8ba2-44ae5312b492 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59752dba564d4b2ea92e65f7faa96596'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:06.615270 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-bd10b9aa-059f-42ee-8ba2-44ae5312b492 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59752dba564d4b2ea92e65f7faa96596'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:06.617765 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1098/2188] 10.222.0.30 () {58 vars in 1058 bytes} [Wed Nov 2 07:31:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:31:06.625830 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-bd10b9aa-059f-42ee-8ba2-44ae5312b492 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59752dba564d4b2ea92e65f7faa96596'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:06.627657 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0559f-669a-4499-9057-9c4c667c3b85 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:06.639143 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1091/2189] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:31:06 2022] DELETE /identity/v3/users/59752dba564d4b2ea92e65f7faa96596 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:06.647859 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0559f-669a-4499-9057-9c4c667c3b85 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:06.747131 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90aea18-79c7-41d4-bb82-e6caee59fa7a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932d3076fb4b426d8c83d02eb60d8063', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:06.747829 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90aea18-79c7-41d4-bb82-e6caee59fa7a service placement] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:06.748106 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90aea18-79c7-41d4-bb82-e6caee59fa7a service placement] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:06.748550 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90aea18-79c7-41d4-bb82-e6caee59fa7a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:06.750708 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-b90aea18-79c7-41d4-bb82-e6caee59fa7a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:06.765830 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90aea18-79c7-41d4-bb82-e6caee59fa7a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:06.768042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90aea18-79c7-41d4-bb82-e6caee59fa7a service placement] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:06.791339 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1099/2190] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:31:07.225266 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10b9aa-059f-42ee-8ba2-44ae5312b492 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:07.227012 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-bd10b9aa-059f-42ee-8ba2-44ae5312b492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:07.321565 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10b9aa-059f-42ee-8ba2-44ae5312b492 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:07.326261 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645368b-f22a-4168-8a23-46604e472ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:07.326948 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645368b-f22a-4168-8a23-46604e472ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:07.327370 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645368b-f22a-4168-8a23-46604e472ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:07.327641 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645368b-f22a-4168-8a23-46604e472ea3 admin admin] PATH_INFO: `/v3/projects/08f287e1b9084b668b8862d28a31c268` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:07.328770 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c645368b-f22a-4168-8a23-46604e472ea3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08f287e1b9084b668b8862d28a31c268)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:07.331112 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c645368b-f22a-4168-8a23-46604e472ea3 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:07.338106 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:07.388857 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c645368b-f22a-4168-8a23-46604e472ea3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08f287e1b9084b668b8862d28a31c268', 'request_id': 'req-c645368b-f22a-4168-8a23-46604e472ea3'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:07.395500 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c645368b-f22a-4168-8a23-46604e472ea3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08f287e1b9084b668b8862d28a31c268', 'request_id': 'req-c645368b-f22a-4168-8a23-46604e472ea3'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:07.401306 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c645368b-f22a-4168-8a23-46604e472ea3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08f287e1b9084b668b8862d28a31c268', 'request_id': 'req-c645368b-f22a-4168-8a23-46604e472ea3'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:07.417796 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c645368b-f22a-4168-8a23-46604e472ea3 admin admin] The token cache is being invalidate because project 08f287e1b9084b668b8862d28a31c268 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:31:07.418121 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c645368b-f22a-4168-8a23-46604e472ea3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:07.418789 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c645368b-f22a-4168-8a23-46604e472ea3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:31:07.420309 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1092/2191] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:31:07 2022] DELETE /identity/v3/projects/08f287e1b9084b668b8862d28a31c268 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:31:07.425635 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67838dcc-4c90-45e9-86ac-28a046f93a79 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:07.426136 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67838dcc-4c90-45e9-86ac-28a046f93a79 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:07.426252 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67838dcc-4c90-45e9-86ac-28a046f93a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:07.445871 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-67838dcc-4c90-45e9-86ac-28a046f93a79 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:07.447199 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-67838dcc-4c90-45e9-86ac-28a046f93a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:07.510895 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1100/2192] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:31:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:07.520038 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb78926-b974-465e-839e-d697732c5c4b None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:07.520320 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb78926-b974-465e-839e-d697732c5c4b None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:07.520590 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb78926-b974-465e-839e-d697732c5c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:07.542417 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-afb78926-b974-465e-839e-d697732c5c4b None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:07.543823 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-afb78926-b974-465e-839e-d697732c5c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:07.553328 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1093/2193] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:31:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:07.557573 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67838dcc-4c90-45e9-86ac-28a046f93a79 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:07.567853 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67838dcc-4c90-45e9-86ac-28a046f93a79 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:07.572614 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8c5a46-59e5-4a6b-a58b-cab2f0d62ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:07.573319 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c5a46-59e5-4a6b-a58b-cab2f0d62ecc admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:07.573599 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c5a46-59e5-4a6b-a58b-cab2f0d62ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:07.573871 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c5a46-59e5-4a6b-a58b-cab2f0d62ecc admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:07.574523 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8c5a46-59e5-4a6b-a58b-cab2f0d62ecc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:31:07.574860 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8c5a46-59e5-4a6b-a58b-cab2f0d62ecc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:07.576467 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8c5a46-59e5-4a6b-a58b-cab2f0d62ecc admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:07.596396 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1101/2194] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:31:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:31:07.597667 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb78926-b974-465e-839e-d697732c5c4b None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:07.612105 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb78926-b974-465e-839e-d697732c5c4b None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:07.617540 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b667c-15e5-41f4-8142-9de575465e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:07.618264 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b667c-15e5-41f4-8142-9de575465e4a admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:07.618635 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b667c-15e5-41f4-8142-9de575465e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:07.619078 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b667c-15e5-41f4-8142-9de575465e4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:07.619861 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b667c-15e5-41f4-8142-9de575465e4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:07.621946 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b667c-15e5-41f4-8142-9de575465e4a admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:07.654579 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1094/2195] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:31:07 2022] POST /identity/v3/projects => generated 363 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:31:07.658879 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8c5a46-59e5-4a6b-a58b-cab2f0d62ecc admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:07.669179 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8c5a46-59e5-4a6b-a58b-cab2f0d62ecc admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:07.732418 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:07.778957 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6ebee-2808-4567-b05e-4db1d97cd7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:07.779599 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6ebee-2808-4567-b05e-4db1d97cd7aa admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:07.779883 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6ebee-2808-4567-b05e-4db1d97cd7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:07.780156 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6ebee-2808-4567-b05e-4db1d97cd7aa admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:07.782122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e6ebee-2808-4567-b05e-4db1d97cd7aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:07.784150 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e6ebee-2808-4567-b05e-4db1d97cd7aa admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:07.810604 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1102/2196] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:31:07 2022] POST /identity/v3/users => generated 282 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:31:07.814331 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b667c-15e5-41f4-8142-9de575465e4a admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:07.825308 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b667c-15e5-41f4-8142-9de575465e4a admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:07.829982 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be14d9-ef33-4299-b2b0-ff08f0e51fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:07.830723 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be14d9-ef33-4299-b2b0-ff08f0e51fe4 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:07.831131 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be14d9-ef33-4299-b2b0-ff08f0e51fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:07.831410 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be14d9-ef33-4299-b2b0-ff08f0e51fe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:07.832162 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be14d9-ef33-4299-b2b0-ff08f0e51fe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:07.833809 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be14d9-ef33-4299-b2b0-ff08f0e51fe4 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:07.848317 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1095/2197] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:31:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:31:07.850023 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6ebee-2808-4567-b05e-4db1d97cd7aa admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:07.861007 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6ebee-2808-4567-b05e-4db1d97cd7aa admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:07.866194 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e3a82-0dcb-47bd-9bac-619ff86bb984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:07.866998 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e3a82-0dcb-47bd-9bac-619ff86bb984 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:07.867339 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e3a82-0dcb-47bd-9bac-619ff86bb984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:07.867693 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e3a82-0dcb-47bd-9bac-619ff86bb984 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:07.868402 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e3a82-0dcb-47bd-9bac-619ff86bb984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:07.870087 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e3a82-0dcb-47bd-9bac-619ff86bb984 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:07.882538 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1103/2198] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:31:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:31:07.886696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be14d9-ef33-4299-b2b0-ff08f0e51fe4 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:07.897170 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be14d9-ef33-4299-b2b0-ff08f0e51fe4 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:07.902436 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f56cc4-c87d-4b89-8f88-81f8f57c53bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:07.903223 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f56cc4-c87d-4b89-8f88-81f8f57c53bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:07.903586 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f56cc4-c87d-4b89-8f88-81f8f57c53bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:07.903925 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f56cc4-c87d-4b89-8f88-81f8f57c53bc admin admin] PATH_INFO: `/v3/projects/6a4159c292c341fe937ec60aa20e770b/users/7fdd1d75f9174de18e0a9724f8428870/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:07.920354 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f56cc4-c87d-4b89-8f88-81f8f57c53bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a4159c292c341fe937ec60aa20e770b, user_id=7fdd1d75f9174de18e0a9724f8428870, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:07.923767 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f56cc4-c87d-4b89-8f88-81f8f57c53bc admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:07.951656 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1096/2199] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:31:07 2022] PUT /identity/v3/projects/6a4159c292c341fe937ec60aa20e770b/users/7fdd1d75f9174de18e0a9724f8428870/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:31:07.960916 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218e164-f21e-4465-a6d3-5be1a3d90141 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:07.961225 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218e164-f21e-4465-a6d3-5be1a3d90141 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:07.961470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218e164-f21e-4465-a6d3-5be1a3d90141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:08.035220 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-f218e164-f21e-4465-a6d3-5be1a3d90141 None None] MFA Rules not processed for user `7fdd1d75f9174de18e0a9724f8428870`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:31:08.036689 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-f218e164-f21e-4465-a6d3-5be1a3d90141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:08.131406 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1104/2200] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:31:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:31:08.144237 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f56cc4-c87d-4b89-8f88-81f8f57c53bc admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:08.146834 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-a6f56cc4-c87d-4b89-8f88-81f8f57c53bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:08.243857 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f56cc4-c87d-4b89-8f88-81f8f57c53bc admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:08.248668 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8cd90f-efd1-40a7-b03d-77eeae21d7ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:08.249773 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8cd90f-efd1-40a7-b03d-77eeae21d7ba service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:08.249773 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8cd90f-efd1-40a7-b03d-77eeae21d7ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:08.250110 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8cd90f-efd1-40a7-b03d-77eeae21d7ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:08.266023 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8cd90f-efd1-40a7-b03d-77eeae21d7ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:08.268056 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8cd90f-efd1-40a7-b03d-77eeae21d7ba service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:08.281788 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1097/2201] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:31:18.124884 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=45,keystone:INSERT=3 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:18.277016 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=75,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:27.287140 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218e164-f21e-4465-a6d3-5be1a3d90141 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:27.288636 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-f218e164-f21e-4465-a6d3-5be1a3d90141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:27.351377 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218e164-f21e-4465-a6d3-5be1a3d90141 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:27.355536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5f78b-bb02-4f23-902f-e8bda6cc1b79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:27.356151 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5f78b-bb02-4f23-902f-e8bda6cc1b79 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:27.356438 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5f78b-bb02-4f23-902f-e8bda6cc1b79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:27.356703 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5f78b-bb02-4f23-902f-e8bda6cc1b79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:27.358459 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-73d5f78b-bb02-4f23-902f-e8bda6cc1b79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:27.430785 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5f78b-bb02-4f23-902f-e8bda6cc1b79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:27.432317 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5f78b-bb02-4f23-902f-e8bda6cc1b79 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:27.443279 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1105/2202] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:27 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:31:37.438386 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=50 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:31:45.729744 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8cd90f-efd1-40a7-b03d-77eeae21d7ba service cinder] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:31:45.739881 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8cd90f-efd1-40a7-b03d-77eeae21d7ba service cinder] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:31:45.744353 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0384428-a13a-444f-9d18-81cdc075f0e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:31:45.745054 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0384428-a13a-444f-9d18-81cdc075f0e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:31:45.745327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0384428-a13a-444f-9d18-81cdc075f0e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:31:45.745587 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0384428-a13a-444f-9d18-81cdc075f0e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:31:45.747587 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-b0384428-a13a-444f-9d18-81cdc075f0e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:31:45.818695 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0384428-a13a-444f-9d18-81cdc075f0e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:31:45.820040 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0384428-a13a-444f-9d18-81cdc075f0e5 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:31:45.830148 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1098/2203] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:31:45 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:31:55.825780 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=30 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:32:01.771945 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5f78b-bb02-4f23-902f-e8bda6cc1b79 service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:32:01.840193 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5f78b-bb02-4f23-902f-e8bda6cc1b79 service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:32:01.846174 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b72afa-0755-4628-918a-3c0d3cb1dd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:32:01.846846 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b72afa-0755-4628-918a-3c0d3cb1dd72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:01.847124 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b72afa-0755-4628-918a-3c0d3cb1dd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:01.847395 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b72afa-0755-4628-918a-3c0d3cb1dd72 admin admin] PATH_INFO: `/v3/users/7fdd1d75f9174de18e0a9724f8428870` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:01.848619 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b72afa-0755-4628-918a-3c0d3cb1dd72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fdd1d75f9174de18e0a9724f8428870)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:32:01.850766 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b72afa-0755-4628-918a-3c0d3cb1dd72 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:32:01.901768 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-e8b72afa-0755-4628-918a-3c0d3cb1dd72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fdd1d75f9174de18e0a9724f8428870'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:32:01.910058 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-e8b72afa-0755-4628-918a-3c0d3cb1dd72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fdd1d75f9174de18e0a9724f8428870'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:32:01.910058 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-e8b72afa-0755-4628-918a-3c0d3cb1dd72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fdd1d75f9174de18e0a9724f8428870'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:32:01.922447 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-e8b72afa-0755-4628-918a-3c0d3cb1dd72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fdd1d75f9174de18e0a9724f8428870'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:32:01.931672 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-e8b72afa-0755-4628-918a-3c0d3cb1dd72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fdd1d75f9174de18e0a9724f8428870'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:32:01.939175 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1106/2204] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:32:01 2022] DELETE /identity/v3/users/7fdd1d75f9174de18e0a9724f8428870 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:32:01.943932 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500c24c-e12d-45c9-99fc-e1a04c143441 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:01.944446 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500c24c-e12d-45c9-99fc-e1a04c143441 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:01.944446 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500c24c-e12d-45c9-99fc-e1a04c143441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:01.963937 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-7500c24c-e12d-45c9-99fc-e1a04c143441 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:32:01.965121 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-7500c24c-e12d-45c9-99fc-e1a04c143441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:32:02.026590 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1099/2205] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:32:02.037034 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b72afa-0755-4628-918a-3c0d3cb1dd72 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:32:02.038589 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-e8b72afa-0755-4628-918a-3c0d3cb1dd72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:32:02.102726 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b72afa-0755-4628-918a-3c0d3cb1dd72 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:32:02.106802 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4558beb-713c-442b-a304-173a348788d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:32:02.107428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4558beb-713c-442b-a304-173a348788d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:02.107705 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4558beb-713c-442b-a304-173a348788d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:02.107975 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4558beb-713c-442b-a304-173a348788d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:02.117728 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4558beb-713c-442b-a304-173a348788d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:32:02.119733 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4558beb-713c-442b-a304-173a348788d7 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:32:02.131206 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1107/2206] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:32:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:32:02.408678 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500c24c-e12d-45c9-99fc-e1a04c143441 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:32:02.410094 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-7500c24c-e12d-45c9-99fc-e1a04c143441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:32:02.422063 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500c24c-e12d-45c9-99fc-e1a04c143441 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:32:02.426760 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab69ca-214b-4bf9-a9d7-c84e50ec51ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:32:02.428134 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab69ca-214b-4bf9-a9d7-c84e50ec51ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:02.428594 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab69ca-214b-4bf9-a9d7-c84e50ec51ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:02.428853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab69ca-214b-4bf9-a9d7-c84e50ec51ca admin admin] PATH_INFO: `/v3/projects/6a4159c292c341fe937ec60aa20e770b` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:02.430061 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efab69ca-214b-4bf9-a9d7-c84e50ec51ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a4159c292c341fe937ec60aa20e770b)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:32:02.432148 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efab69ca-214b-4bf9-a9d7-c84e50ec51ca admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:32:02.484737 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-efab69ca-214b-4bf9-a9d7-c84e50ec51ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a4159c292c341fe937ec60aa20e770b', 'request_id': 'req-efab69ca-214b-4bf9-a9d7-c84e50ec51ca'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:32:02.491017 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-efab69ca-214b-4bf9-a9d7-c84e50ec51ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a4159c292c341fe937ec60aa20e770b', 'request_id': 'req-efab69ca-214b-4bf9-a9d7-c84e50ec51ca'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:32:02.496460 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-efab69ca-214b-4bf9-a9d7-c84e50ec51ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a4159c292c341fe937ec60aa20e770b', 'request_id': 'req-efab69ca-214b-4bf9-a9d7-c84e50ec51ca'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:32:02.512151 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-efab69ca-214b-4bf9-a9d7-c84e50ec51ca admin admin] The token cache is being invalidate because project 6a4159c292c341fe937ec60aa20e770b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:32:02.512529 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-efab69ca-214b-4bf9-a9d7-c84e50ec51ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:32:02.513164 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-efab69ca-214b-4bf9-a9d7-c84e50ec51ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:32:02.514726 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1100/2207] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:32:02 2022] DELETE /identity/v3/projects/6a4159c292c341fe937ec60aa20e770b => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:32:12.126516 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:32:12.509303 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:32:16.942349 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36a961-b8cf-437a-b58b-12f8a2c56cde None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:16.942917 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36a961-b8cf-437a-b58b-12f8a2c56cde None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:16.942917 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36a961-b8cf-437a-b58b-12f8a2c56cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:16.963623 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-ac36a961-b8cf-437a-b58b-12f8a2c56cde None None] MFA Rules not processed for user `2ffdd4e961244bffa2b22490956ab51f`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:32:16.964620 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-ac36a961-b8cf-437a-b58b-12f8a2c56cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:32:17.022918 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1108/2208] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:32:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:32:27.015849 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=25 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:32:48.610013 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b741c-1bd3-4a0b-acab-b18eb97d95f8 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:48.610562 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b741c-1bd3-4a0b-acab-b18eb97d95f8 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:48.610562 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b741c-1bd3-4a0b-acab-b18eb97d95f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:48.631438 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-a53b741c-1bd3-4a0b-acab-b18eb97d95f8 None None] MFA Rules not processed for user `be586640013949589d2a7b11a7a2f3b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:32:48.632813 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-a53b741c-1bd3-4a0b-acab-b18eb97d95f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:32:48.692389 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1101/2209] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:32:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:32:56.021331 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac36a961-b8cf-437a-b58b-12f8a2c56cde None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:32:56.022990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-ac36a961-b8cf-437a-b58b-12f8a2c56cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:32:56.084210 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac36a961-b8cf-437a-b58b-12f8a2c56cde None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:32:56.087957 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27266bd9-b24d-4ed1-9e20-5d1c8298a270 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:32:56.088570 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27266bd9-b24d-4ed1-9e20-5d1c8298a270 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:56.088843 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27266bd9-b24d-4ed1-9e20-5d1c8298a270 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:56.089109 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27266bd9-b24d-4ed1-9e20-5d1c8298a270 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:56.090855 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-27266bd9-b24d-4ed1-9e20-5d1c8298a270 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:32:56.194027 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27266bd9-b24d-4ed1-9e20-5d1c8298a270 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:32:56.195907 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27266bd9-b24d-4ed1-9e20-5d1c8298a270 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:32:56.218866 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1109/2210] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:32:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:32:56.238984 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b741c-1bd3-4a0b-acab-b18eb97d95f8 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:32:56.241312 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-a53b741c-1bd3-4a0b-acab-b18eb97d95f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:32:56.255431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b741c-1bd3-4a0b-acab-b18eb97d95f8 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:32:56.260869 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80694a8-ff36-4936-82a4-c1d878ab8414 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:32:56.261703 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80694a8-ff36-4936-82a4-c1d878ab8414 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:56.262153 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80694a8-ff36-4936-82a4-c1d878ab8414 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:56.262578 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80694a8-ff36-4936-82a4-c1d878ab8414 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:56.264938 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-b80694a8-ff36-4936-82a4-c1d878ab8414 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:32:56.295925 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80694a8-ff36-4936-82a4-c1d878ab8414 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:32:56.297365 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80694a8-ff36-4936-82a4-c1d878ab8414 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:32:56.332748 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1102/2211] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:32:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:32:58.659452 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408da14b-7fef-4ea0-8209-139f46ec4411 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:58.659938 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408da14b-7fef-4ea0-8209-139f46ec4411 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:58.660042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408da14b-7fef-4ea0-8209-139f46ec4411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:58.679157 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-408da14b-7fef-4ea0-8209-139f46ec4411 None None] MFA Rules not processed for user `364555ec385c4dbb91907fb9c5116d2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:32:58.680368 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-408da14b-7fef-4ea0-8209-139f46ec4411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:32:58.737887 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1110/2212] 10.222.0.30 () {58 vars in 967 bytes} [Wed Nov 2 07:32:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:32:58.781624 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80694a8-ff36-4936-82a4-c1d878ab8414 service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:32:58.783103 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-b80694a8-ff36-4936-82a4-c1d878ab8414 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:32:58.861965 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80694a8-ff36-4936-82a4-c1d878ab8414 service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:32:58.869345 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3a573-d40e-4860-916a-9585c5ba0875 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '836c4044aaf74de391219289b57d2956', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:32:58.869946 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3a573-d40e-4860-916a-9585c5ba0875 service swift] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:32:58.870214 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3a573-d40e-4860-916a-9585c5ba0875 service swift] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:32:58.870475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3a573-d40e-4860-916a-9585c5ba0875 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:32:58.880636 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3a573-d40e-4860-916a-9585c5ba0875 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:32:58.882221 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3a573-d40e-4860-916a-9585c5ba0875 service swift] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:32:58.891170 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1103/2213] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Nov 2 07:32:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:33:06.149053 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a012274-83b7-4a6d-8be0-808ec150e468 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:33:06.149912 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a012274-83b7-4a6d-8be0-808ec150e468 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:33:06.150856 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a012274-83b7-4a6d-8be0-808ec150e468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:33:06.172121 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-5a012274-83b7-4a6d-8be0-808ec150e468 None None] MFA Rules not processed for user `e4f9bb9b60a246f3a84a4d4618c3f527`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:33:06.173887 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-5a012274-83b7-4a6d-8be0-808ec150e468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:06.225724 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:33:06.252005 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1111/2214] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:33:06 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:33:07.165286 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3a573-d40e-4860-916a-9585c5ba0875 service swift] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:33:07.166781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-6eb3a573-d40e-4860-916a-9585c5ba0875 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:07.222243 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a012274-83b7-4a6d-8be0-808ec150e468 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:33:07.223749 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-5a012274-83b7-4a6d-8be0-808ec150e468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:07.246843 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3a573-d40e-4860-916a-9585c5ba0875 service swift] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:33:07.252064 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47390316-9cc6-4ae1-9928-863cc5248ddf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932d3076fb4b426d8c83d02eb60d8063', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:33:07.252682 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47390316-9cc6-4ae1-9928-863cc5248ddf service placement] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:33:07.252955 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47390316-9cc6-4ae1-9928-863cc5248ddf service placement] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:33:07.253224 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47390316-9cc6-4ae1-9928-863cc5248ddf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:33:07.255625 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-47390316-9cc6-4ae1-9928-863cc5248ddf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:07.268371 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47390316-9cc6-4ae1-9928-863cc5248ddf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:33:07.269941 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47390316-9cc6-4ae1-9928-863cc5248ddf service placement] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:33:07.280188 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1104/2215] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:33:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:33:07.295283 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a012274-83b7-4a6d-8be0-808ec150e468 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:33:07.300613 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b6dbb-160f-4029-b890-95fd42aa6a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:33:07.301268 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b6dbb-160f-4029-b890-95fd42aa6a4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:33:07.301539 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b6dbb-160f-4029-b890-95fd42aa6a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:33:07.301809 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b6dbb-160f-4029-b890-95fd42aa6a4e admin admin] PATH_INFO: `/v3/users/25afd7d8fcfe4dcf9133872d69520508` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:33:07.303000 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71b6dbb-160f-4029-b890-95fd42aa6a4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=25afd7d8fcfe4dcf9133872d69520508)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:33:07.304990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71b6dbb-160f-4029-b890-95fd42aa6a4e admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:33:07.354679 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-e71b6dbb-160f-4029-b890-95fd42aa6a4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25afd7d8fcfe4dcf9133872d69520508'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:33:07.361661 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-e71b6dbb-160f-4029-b890-95fd42aa6a4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25afd7d8fcfe4dcf9133872d69520508'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:33:07.362346 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-e71b6dbb-160f-4029-b890-95fd42aa6a4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25afd7d8fcfe4dcf9133872d69520508'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:33:07.373992 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-e71b6dbb-160f-4029-b890-95fd42aa6a4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25afd7d8fcfe4dcf9133872d69520508'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:33:07.383608 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-e71b6dbb-160f-4029-b890-95fd42aa6a4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25afd7d8fcfe4dcf9133872d69520508'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:33:07.389720 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1112/2216] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:33:07 2022] DELETE /identity/v3/users/25afd7d8fcfe4dcf9133872d69520508 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:33:07.576828 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47390316-9cc6-4ae1-9928-863cc5248ddf service placement] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:33:07.588146 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47390316-9cc6-4ae1-9928-863cc5248ddf service placement] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:33:07.597726 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:33:07.660010 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54127055-c61f-4084-8c88-23fee5686996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:33:07.660748 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54127055-c61f-4084-8c88-23fee5686996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:33:07.661080 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54127055-c61f-4084-8c88-23fee5686996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:33:07.661354 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54127055-c61f-4084-8c88-23fee5686996 admin admin] PATH_INFO: `/v3/projects/4866e41079fb4b0298212f80dc15399c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:33:07.662600 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54127055-c61f-4084-8c88-23fee5686996 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4866e41079fb4b0298212f80dc15399c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:33:07.664594 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54127055-c61f-4084-8c88-23fee5686996 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:33:07.715534 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-54127055-c61f-4084-8c88-23fee5686996 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4866e41079fb4b0298212f80dc15399c', 'request_id': 'req-54127055-c61f-4084-8c88-23fee5686996'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:33:07.722529 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-54127055-c61f-4084-8c88-23fee5686996 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4866e41079fb4b0298212f80dc15399c', 'request_id': 'req-54127055-c61f-4084-8c88-23fee5686996'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:33:07.728370 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-54127055-c61f-4084-8c88-23fee5686996 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4866e41079fb4b0298212f80dc15399c', 'request_id': 'req-54127055-c61f-4084-8c88-23fee5686996'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:33:07.742514 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-54127055-c61f-4084-8c88-23fee5686996 admin admin] The token cache is being invalidate because project 4866e41079fb4b0298212f80dc15399c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:33:07.742861 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-54127055-c61f-4084-8c88-23fee5686996 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:33:07.743611 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-54127055-c61f-4084-8c88-23fee5686996 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:33:07.745387 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1105/2217] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:33:07 2022] DELETE /identity/v3/projects/4866e41079fb4b0298212f80dc15399c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:33:07.749133 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b6dbb-160f-4029-b890-95fd42aa6a4e admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:33:07.750668 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-e71b6dbb-160f-4029-b890-95fd42aa6a4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:07.764995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b6dbb-160f-4029-b890-95fd42aa6a4e admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:33:07.825147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3dc2d6-1c0d-4398-b486-109b70dd09c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:33:07.825791 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3dc2d6-1c0d-4398-b486-109b70dd09c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:33:07.826070 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3dc2d6-1c0d-4398-b486-109b70dd09c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:33:07.826339 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3dc2d6-1c0d-4398-b486-109b70dd09c7 admin admin] PATH_INFO: `/v3/users/9adce6fc87a840e2baa33bb60d3ddcfd` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:33:07.827708 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3dc2d6-1c0d-4398-b486-109b70dd09c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9adce6fc87a840e2baa33bb60d3ddcfd)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:33:07.829786 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3dc2d6-1c0d-4398-b486-109b70dd09c7 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:33:07.842518 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:33:07.884110 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-ea3dc2d6-1c0d-4398-b486-109b70dd09c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9adce6fc87a840e2baa33bb60d3ddcfd'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:33:07.891266 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-ea3dc2d6-1c0d-4398-b486-109b70dd09c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9adce6fc87a840e2baa33bb60d3ddcfd'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:33:07.891920 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-ea3dc2d6-1c0d-4398-b486-109b70dd09c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9adce6fc87a840e2baa33bb60d3ddcfd'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:33:07.904297 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-ea3dc2d6-1c0d-4398-b486-109b70dd09c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9adce6fc87a840e2baa33bb60d3ddcfd'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:33:07.913076 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-ea3dc2d6-1c0d-4398-b486-109b70dd09c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9adce6fc87a840e2baa33bb60d3ddcfd'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:33:07.919578 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1113/2218] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:33:07 2022] DELETE /identity/v3/users/9adce6fc87a840e2baa33bb60d3ddcfd => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:33:08.201462 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54127055-c61f-4084-8c88-23fee5686996 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:33:08.203352 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-54127055-c61f-4084-8c88-23fee5686996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:08.221053 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54127055-c61f-4084-8c88-23fee5686996 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:33:08.278607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509a224-d14e-4ccd-a283-403f9083f3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:33:08.279186 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509a224-d14e-4ccd-a283-403f9083f3f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:33:08.279454 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509a224-d14e-4ccd-a283-403f9083f3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:33:08.279718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509a224-d14e-4ccd-a283-403f9083f3f4 admin admin] PATH_INFO: `/v3/projects/890bb2756d444dbf982b606eea0e37c5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:33:08.280827 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509a224-d14e-4ccd-a283-403f9083f3f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=890bb2756d444dbf982b606eea0e37c5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:33:08.282792 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509a224-d14e-4ccd-a283-403f9083f3f4 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:33:08.326579 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:33:08.330706 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-f509a224-d14e-4ccd-a283-403f9083f3f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '890bb2756d444dbf982b606eea0e37c5', 'request_id': 'req-f509a224-d14e-4ccd-a283-403f9083f3f4'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:33:08.337554 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-f509a224-d14e-4ccd-a283-403f9083f3f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '890bb2756d444dbf982b606eea0e37c5', 'request_id': 'req-f509a224-d14e-4ccd-a283-403f9083f3f4'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:33:08.343607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-f509a224-d14e-4ccd-a283-403f9083f3f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '890bb2756d444dbf982b606eea0e37c5', 'request_id': 'req-f509a224-d14e-4ccd-a283-403f9083f3f4'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:33:08.356738 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-f509a224-d14e-4ccd-a283-403f9083f3f4 admin admin] The token cache is being invalidate because project 890bb2756d444dbf982b606eea0e37c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:33:08.357022 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-f509a224-d14e-4ccd-a283-403f9083f3f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:33:08.357649 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-f509a224-d14e-4ccd-a283-403f9083f3f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:33:08.359192 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1106/2219] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:33:08 2022] DELETE /identity/v3/projects/890bb2756d444dbf982b606eea0e37c5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:33:09.108462 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3dc2d6-1c0d-4398-b486-109b70dd09c7 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:33:09.111157 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-ea3dc2d6-1c0d-4398-b486-109b70dd09c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:09.194500 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3dc2d6-1c0d-4398-b486-109b70dd09c7 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:33:09.260544 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfce789-3051-44e5-baa0-d8acd47f6737 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:33:09.261162 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfce789-3051-44e5-baa0-d8acd47f6737 service nova] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:33:09.261443 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfce789-3051-44e5-baa0-d8acd47f6737 service nova] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:33:09.261712 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfce789-3051-44e5-baa0-d8acd47f6737 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:33:09.263443 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-5dfce789-3051-44e5-baa0-d8acd47f6737 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:09.289398 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:33:09.327777 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfce789-3051-44e5-baa0-d8acd47f6737 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:33:09.329493 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfce789-3051-44e5-baa0-d8acd47f6737 service nova] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:33:09.339350 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1114/2220] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:33:09 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:33:18.353564 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=8,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:33:19.335081 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=16 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:33:21.050044 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509a224-d14e-4ccd-a283-403f9083f3f4 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:33:21.051715 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-f509a224-d14e-4ccd-a283-403f9083f3f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:21.102088 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfce789-3051-44e5-baa0-d8acd47f6737 service nova] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:33:21.103924 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-5dfce789-3051-44e5-baa0-d8acd47f6737 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:21.126448 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509a224-d14e-4ccd-a283-403f9083f3f4 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:33:21.132763 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71779760-f11d-4483-9568-e551f65c66e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932d3076fb4b426d8c83d02eb60d8063', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:33:21.133386 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71779760-f11d-4483-9568-e551f65c66e2 service placement] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:33:21.133663 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71779760-f11d-4483-9568-e551f65c66e2 service placement] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:33:21.133917 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71779760-f11d-4483-9568-e551f65c66e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:33:21.135695 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-71779760-f11d-4483-9568-e551f65c66e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:21.147706 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71779760-f11d-4483-9568-e551f65c66e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:33:21.149226 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71779760-f11d-4483-9568-e551f65c66e2 service placement] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:33:21.159770 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1107/2221] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:33:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:33:21.173334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfce789-3051-44e5-baa0-d8acd47f6737 service nova] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:33:21.178371 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec97de9-3f83-40b4-bc52-e15d16500090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:33:21.179043 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec97de9-3f83-40b4-bc52-e15d16500090 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:33:21.179318 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec97de9-3f83-40b4-bc52-e15d16500090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:33:21.179589 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec97de9-3f83-40b4-bc52-e15d16500090 admin admin] PATH_INFO: `/v3/users/2ffdd4e961244bffa2b22490956ab51f` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:33:21.180721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec97de9-3f83-40b4-bc52-e15d16500090 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ffdd4e961244bffa2b22490956ab51f)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:33:21.182650 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec97de9-3f83-40b4-bc52-e15d16500090 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:33:21.231041 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-7ec97de9-3f83-40b4-bc52-e15d16500090 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ffdd4e961244bffa2b22490956ab51f'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:33:21.239301 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-7ec97de9-3f83-40b4-bc52-e15d16500090 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ffdd4e961244bffa2b22490956ab51f'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:33:21.240050 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-7ec97de9-3f83-40b4-bc52-e15d16500090 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ffdd4e961244bffa2b22490956ab51f'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:33:21.253235 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-7ec97de9-3f83-40b4-bc52-e15d16500090 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ffdd4e961244bffa2b22490956ab51f'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:33:21.263158 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-7ec97de9-3f83-40b4-bc52-e15d16500090 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ffdd4e961244bffa2b22490956ab51f'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:33:21.269300 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1115/2222] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:33:21 2022] DELETE /identity/v3/users/2ffdd4e961244bffa2b22490956ab51f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:33:21.454985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71779760-f11d-4483-9568-e551f65c66e2 service placement] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:33:21.467509 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71779760-f11d-4483-9568-e551f65c66e2 service placement] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:33:21.544805 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef59b8-d21f-45ef-947f-89b8e312569b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:33:21.545456 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef59b8-d21f-45ef-947f-89b8e312569b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:33:21.545732 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef59b8-d21f-45ef-947f-89b8e312569b admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:33:21.545975 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef59b8-d21f-45ef-947f-89b8e312569b admin admin] PATH_INFO: `/v3/projects/8aca24dbea0247549df7bdc687f7e2a7` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:33:21.547333 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ef59b8-d21f-45ef-947f-89b8e312569b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8aca24dbea0247549df7bdc687f7e2a7)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:33:21.549254 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ef59b8-d21f-45ef-947f-89b8e312569b admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:33:21.600556 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-70ef59b8-d21f-45ef-947f-89b8e312569b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8aca24dbea0247549df7bdc687f7e2a7', 'request_id': 'req-70ef59b8-d21f-45ef-947f-89b8e312569b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:33:21.607275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-70ef59b8-d21f-45ef-947f-89b8e312569b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8aca24dbea0247549df7bdc687f7e2a7', 'request_id': 'req-70ef59b8-d21f-45ef-947f-89b8e312569b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:33:21.615513 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-70ef59b8-d21f-45ef-947f-89b8e312569b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8aca24dbea0247549df7bdc687f7e2a7', 'request_id': 'req-70ef59b8-d21f-45ef-947f-89b8e312569b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:33:21.630299 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-70ef59b8-d21f-45ef-947f-89b8e312569b admin admin] The token cache is being invalidate because project 8aca24dbea0247549df7bdc687f7e2a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:33:21.630583 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-70ef59b8-d21f-45ef-947f-89b8e312569b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:33:21.631273 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-70ef59b8-d21f-45ef-947f-89b8e312569b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:33:21.632845 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1108/2223] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:33:21 2022] DELETE /identity/v3/projects/8aca24dbea0247549df7bdc687f7e2a7 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:33:31.266025 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:33:31.627105 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:33:47.725332 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec97de9-3f83-40b4-bc52-e15d16500090 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:33:47.726872 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-7ec97de9-3f83-40b4-bc52-e15d16500090 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:47.804866 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec97de9-3f83-40b4-bc52-e15d16500090 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:33:47.878429 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6d159-a4aa-4949-b37f-0b0f0462840c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:33:47.879092 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6d159-a4aa-4949-b37f-0b0f0462840c service nova] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:33:47.879374 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6d159-a4aa-4949-b37f-0b0f0462840c service nova] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:33:47.879646 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6d159-a4aa-4949-b37f-0b0f0462840c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:33:47.881651 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-b1b6d159-a4aa-4949-b37f-0b0f0462840c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:47.946809 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6d159-a4aa-4949-b37f-0b0f0462840c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:33:47.948414 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6d159-a4aa-4949-b37f-0b0f0462840c service nova] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:33:47.958194 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1116/2224] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:33:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:33:52.929794 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef59b8-d21f-45ef-947f-89b8e312569b admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:33:52.931347 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-70ef59b8-d21f-45ef-947f-89b8e312569b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:52.947807 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef59b8-d21f-45ef-947f-89b8e312569b admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:33:52.953664 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb5bc73-f86d-492d-a076-14677be82ee3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:33:52.954279 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5bc73-f86d-492d-a076-14677be82ee3 service nova] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:33:52.954554 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5bc73-f86d-492d-a076-14677be82ee3 service nova] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:33:52.954858 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5bc73-f86d-492d-a076-14677be82ee3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:33:52.956539 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-1bb5bc73-f86d-492d-a076-14677be82ee3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:52.969553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb5bc73-f86d-492d-a076-14677be82ee3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:33:52.971481 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb5bc73-f86d-492d-a076-14677be82ee3 service nova] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:33:52.983133 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1109/2225] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:33:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:33:57.078475 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65b4d7-050a-4c54-a448-a481e03572e7 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:33:57.078475 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65b4d7-050a-4c54-a448-a481e03572e7 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:33:57.078475 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65b4d7-050a-4c54-a448-a481e03572e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:33:57.102846 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-4a65b4d7-050a-4c54-a448-a481e03572e7 None None] MFA Rules not processed for user `a47b81144eba4283aac610c600e73ed3`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:33:57.104207 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-4a65b4d7-050a-4c54-a448-a481e03572e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:57.138694 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:33:57.166525 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1117/2226] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:33:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:33:57.488548 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb5bc73-f86d-492d-a076-14677be82ee3 service nova] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:33:57.490079 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-1bb5bc73-f86d-492d-a076-14677be82ee3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:57.550427 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb5bc73-f86d-492d-a076-14677be82ee3 service nova] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:33:57.554322 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a64e9-08e9-438d-9c02-9a02047052cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:33:57.554961 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a64e9-08e9-438d-9c02-9a02047052cc service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:33:57.555232 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a64e9-08e9-438d-9c02-9a02047052cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:33:57.555499 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a64e9-08e9-438d-9c02-9a02047052cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:33:57.557403 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-5a6a64e9-08e9-438d-9c02-9a02047052cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:57.567735 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6a64e9-08e9-438d-9c02-9a02047052cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:33:57.569166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6a64e9-08e9-438d-9c02-9a02047052cc service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:33:57.577606 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1110/2227] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:33:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:33:58.055950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65b4d7-050a-4c54-a448-a481e03572e7 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:33:58.057616 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-4a65b4d7-050a-4c54-a448-a481e03572e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:58.136566 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65b4d7-050a-4c54-a448-a481e03572e7 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:33:58.141902 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658c5a2-ef99-4b0b-8818-215c7be89da3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932d3076fb4b426d8c83d02eb60d8063', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:33:58.142502 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658c5a2-ef99-4b0b-8818-215c7be89da3 service placement] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:33:58.142804 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658c5a2-ef99-4b0b-8818-215c7be89da3 service placement] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:33:58.143099 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658c5a2-ef99-4b0b-8818-215c7be89da3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:33:58.144818 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-f658c5a2-ef99-4b0b-8818-215c7be89da3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:33:58.157501 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658c5a2-ef99-4b0b-8818-215c7be89da3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:33:58.159138 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658c5a2-ef99-4b0b-8818-215c7be89da3 service placement] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:33:58.170346 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1118/2228] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:33:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:34:02.707064 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a64e9-08e9-438d-9c02-9a02047052cc service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:02.708779 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-5a6a64e9-08e9-438d-9c02-9a02047052cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:02.720718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a64e9-08e9-438d-9c02-9a02047052cc service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:02.725398 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78d85b8-2c9d-4425-be28-f1001910dfc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:02.726098 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d85b8-2c9d-4425-be28-f1001910dfc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:02.726395 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d85b8-2c9d-4425-be28-f1001910dfc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:02.726746 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d85b8-2c9d-4425-be28-f1001910dfc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:02.728659 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-e78d85b8-2c9d-4425-be28-f1001910dfc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:02.739894 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78d85b8-2c9d-4425-be28-f1001910dfc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:02.741388 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78d85b8-2c9d-4425-be28-f1001910dfc4 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:02.752181 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1111/2229] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:34:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:34:08.165296 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=41 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:10.166750 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658c5a2-ef99-4b0b-8818-215c7be89da3 service placement] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:10.176675 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658c5a2-ef99-4b0b-8818-215c7be89da3 service placement] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:10.190758 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcc825-961e-488f-9646-be190d1a9f03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:10.190758 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcc825-961e-488f-9646-be190d1a9f03 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:10.190758 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcc825-961e-488f-9646-be190d1a9f03 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:10.190758 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcc825-961e-488f-9646-be190d1a9f03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:10.191210 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-e6bcc825-961e-488f-9646-be190d1a9f03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:10.264377 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bcc825-961e-488f-9646-be190d1a9f03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:10.265399 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bcc825-961e-488f-9646-be190d1a9f03 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:10.277279 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1119/2230] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:34:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:34:12.745918 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=38 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:13.969671 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0adf6a-aacd-4d4b-a529-b46b705be7d7 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:13.970300 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0adf6a-aacd-4d4b-a529-b46b705be7d7 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:13.970860 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0adf6a-aacd-4d4b-a529-b46b705be7d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:13.991880 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-6b0adf6a-aacd-4d4b-a529-b46b705be7d7 None None] MFA Rules not processed for user `be586640013949589d2a7b11a7a2f3b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:13.993449 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-6b0adf6a-aacd-4d4b-a529-b46b705be7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:14.017514 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1112/2231] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:34:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:34:14.928169 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcc825-961e-488f-9646-be190d1a9f03 service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:14.930148 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-e6bcc825-961e-488f-9646-be190d1a9f03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:14.953957 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcc825-961e-488f-9646-be190d1a9f03 service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:14.960729 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d027e02-cc78-4a47-8644-e7480bd30c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:14.961573 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d027e02-cc78-4a47-8644-e7480bd30c45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:14.962219 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d027e02-cc78-4a47-8644-e7480bd30c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:14.962978 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d027e02-cc78-4a47-8644-e7480bd30c45 admin admin] PATH_INFO: `/v3/users/e4f9bb9b60a246f3a84a4d4618c3f527` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:14.964312 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d027e02-cc78-4a47-8644-e7480bd30c45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4f9bb9b60a246f3a84a4d4618c3f527)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:14.966566 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d027e02-cc78-4a47-8644-e7480bd30c45 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:15.030801 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0d027e02-cc78-4a47-8644-e7480bd30c45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4f9bb9b60a246f3a84a4d4618c3f527'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:15.042922 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0d027e02-cc78-4a47-8644-e7480bd30c45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4f9bb9b60a246f3a84a4d4618c3f527'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:15.054341 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0d027e02-cc78-4a47-8644-e7480bd30c45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4f9bb9b60a246f3a84a4d4618c3f527'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:15.069065 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0d027e02-cc78-4a47-8644-e7480bd30c45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4f9bb9b60a246f3a84a4d4618c3f527'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:15.078501 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0d027e02-cc78-4a47-8644-e7480bd30c45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4f9bb9b60a246f3a84a4d4618c3f527'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:15.085841 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1120/2232] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:34:14 2022] DELETE /identity/v3/users/e4f9bb9b60a246f3a84a4d4618c3f527 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:34:15.261514 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0adf6a-aacd-4d4b-a529-b46b705be7d7 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:15.284809 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0adf6a-aacd-4d4b-a529-b46b705be7d7 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:15.352557 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae808832-d5a6-482e-bfe2-c4be2ff5350f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:15.353425 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae808832-d5a6-482e-bfe2-c4be2ff5350f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:15.353752 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae808832-d5a6-482e-bfe2-c4be2ff5350f admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:15.353945 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae808832-d5a6-482e-bfe2-c4be2ff5350f admin admin] PATH_INFO: `/v3/projects/1dfbab8dc316469880ccb755f4d91a32` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:15.355259 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae808832-d5a6-482e-bfe2-c4be2ff5350f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dfbab8dc316469880ccb755f4d91a32)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:15.357111 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae808832-d5a6-482e-bfe2-c4be2ff5350f admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:15.405654 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-ae808832-d5a6-482e-bfe2-c4be2ff5350f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dfbab8dc316469880ccb755f4d91a32', 'request_id': 'req-ae808832-d5a6-482e-bfe2-c4be2ff5350f'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:15.412063 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-ae808832-d5a6-482e-bfe2-c4be2ff5350f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dfbab8dc316469880ccb755f4d91a32', 'request_id': 'req-ae808832-d5a6-482e-bfe2-c4be2ff5350f'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:15.418237 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-ae808832-d5a6-482e-bfe2-c4be2ff5350f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dfbab8dc316469880ccb755f4d91a32', 'request_id': 'req-ae808832-d5a6-482e-bfe2-c4be2ff5350f'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:15.431127 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-ae808832-d5a6-482e-bfe2-c4be2ff5350f admin admin] The token cache is being invalidate because project 1dfbab8dc316469880ccb755f4d91a32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:34:15.431412 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-ae808832-d5a6-482e-bfe2-c4be2ff5350f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:15.432030 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-ae808832-d5a6-482e-bfe2-c4be2ff5350f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:34:15.433522 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1113/2233] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:34:15 2022] DELETE /identity/v3/projects/1dfbab8dc316469880ccb755f4d91a32 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:34:15.438891 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80362c7f-9ed7-4df5-9442-0863f1d69a3b None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:15.439213 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80362c7f-9ed7-4df5-9442-0863f1d69a3b None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:15.439491 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80362c7f-9ed7-4df5-9442-0863f1d69a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:15.459656 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-80362c7f-9ed7-4df5-9442-0863f1d69a3b None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:15.460788 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-80362c7f-9ed7-4df5-9442-0863f1d69a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:15.522826 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1121/2234] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:34:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:34:15.531504 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f1ac7-ec83-43de-add5-ad252ed92c86 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:15.531789 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f1ac7-ec83-43de-add5-ad252ed92c86 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:15.532056 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f1ac7-ec83-43de-add5-ad252ed92c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:15.552766 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-ce0f1ac7-ec83-43de-add5-ad252ed92c86 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:15.553990 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-ce0f1ac7-ec83-43de-add5-ad252ed92c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:15.562981 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1114/2235] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:34:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:34:15.568232 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80362c7f-9ed7-4df5-9442-0863f1d69a3b None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:15.579423 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80362c7f-9ed7-4df5-9442-0863f1d69a3b None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:15.584386 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd9139-67d5-4ecd-9f8a-e71b6ffe424b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:15.585042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd9139-67d5-4ecd-9f8a-e71b6ffe424b admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:15.585318 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd9139-67d5-4ecd-9f8a-e71b6ffe424b admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:15.585588 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd9139-67d5-4ecd-9f8a-e71b6ffe424b admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:15.586241 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd9139-67d5-4ecd-9f8a-e71b6ffe424b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:34:15.586542 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd9139-67d5-4ecd-9f8a-e71b6ffe424b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:15.588205 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd9139-67d5-4ecd-9f8a-e71b6ffe424b admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:15.589868 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:15.604653 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1122/2236] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:34:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:34:15.608240 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f1ac7-ec83-43de-add5-ad252ed92c86 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:15.617722 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f1ac7-ec83-43de-add5-ad252ed92c86 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:15.622639 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422a87f-5509-442a-abc8-4f18d6a3f538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:15.623752 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422a87f-5509-442a-abc8-4f18d6a3f538 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:15.624028 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422a87f-5509-442a-abc8-4f18d6a3f538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:15.624295 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422a87f-5509-442a-abc8-4f18d6a3f538 admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:15.624907 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c422a87f-5509-442a-abc8-4f18d6a3f538 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:15.626861 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c422a87f-5509-442a-abc8-4f18d6a3f538 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:15.655619 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1115/2237] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:34:15 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:34:15.659945 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd9139-67d5-4ecd-9f8a-e71b6ffe424b admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:15.670531 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd9139-67d5-4ecd-9f8a-e71b6ffe424b admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:15.733419 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6aaa99-6b01-429b-b3b7-96112a2783de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:15.734019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6aaa99-6b01-429b-b3b7-96112a2783de admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:15.734302 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6aaa99-6b01-429b-b3b7-96112a2783de admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:15.734558 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6aaa99-6b01-429b-b3b7-96112a2783de admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:15.735222 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6aaa99-6b01-429b-b3b7-96112a2783de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:15.737969 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6aaa99-6b01-429b-b3b7-96112a2783de admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:15.765996 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1123/2238] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:34:15 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:34:15.767313 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422a87f-5509-442a-abc8-4f18d6a3f538 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:15.780277 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422a87f-5509-442a-abc8-4f18d6a3f538 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:15.785957 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf95452-0641-479b-8bc5-42649d61b4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:15.787211 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf95452-0641-479b-8bc5-42649d61b4d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:15.787718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf95452-0641-479b-8bc5-42649d61b4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:15.787718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf95452-0641-479b-8bc5-42649d61b4d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:15.788750 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf95452-0641-479b-8bc5-42649d61b4d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:15.790536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf95452-0641-479b-8bc5-42649d61b4d4 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:15.803205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1116/2239] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:34:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:34:15.807282 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6aaa99-6b01-429b-b3b7-96112a2783de admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:15.817988 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6aaa99-6b01-429b-b3b7-96112a2783de admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:15.823912 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da6050-2f94-4afb-b5c0-985940afaa17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:15.824531 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da6050-2f94-4afb-b5c0-985940afaa17 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:15.825022 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da6050-2f94-4afb-b5c0-985940afaa17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:15.825184 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da6050-2f94-4afb-b5c0-985940afaa17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:15.825825 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da6050-2f94-4afb-b5c0-985940afaa17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:15.827750 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da6050-2f94-4afb-b5c0-985940afaa17 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:15.842299 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1124/2240] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:34:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:34:15.844851 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf95452-0641-479b-8bc5-42649d61b4d4 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:15.854865 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf95452-0641-479b-8bc5-42649d61b4d4 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:15.860235 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f90ab-3653-4d99-b073-8efbf8ec47cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:15.860384 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f90ab-3653-4d99-b073-8efbf8ec47cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:15.860654 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f90ab-3653-4d99-b073-8efbf8ec47cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:15.860920 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f90ab-3653-4d99-b073-8efbf8ec47cd admin admin] PATH_INFO: `/v3/projects/f1d002fd08c941a798ee709ace20f420/users/0e1da9439ab44ce399e72bea608c2c29/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:15.876488 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f90ab-3653-4d99-b073-8efbf8ec47cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1d002fd08c941a798ee709ace20f420, user_id=0e1da9439ab44ce399e72bea608c2c29, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:15.879280 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f90ab-3653-4d99-b073-8efbf8ec47cd admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:15.895652 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1117/2241] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:34:15 2022] PUT /identity/v3/projects/f1d002fd08c941a798ee709ace20f420/users/0e1da9439ab44ce399e72bea608c2c29/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:34:15.900910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b8b67-dc50-481f-b32a-5278297ee20c None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:15.901380 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b8b67-dc50-481f-b32a-5278297ee20c None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:15.901826 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b8b67-dc50-481f-b32a-5278297ee20c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:15.924018 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-c88b8b67-dc50-481f-b32a-5278297ee20c None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:15.925214 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-c88b8b67-dc50-481f-b32a-5278297ee20c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:15.986607 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1125/2242] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:34:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:34:15.999629 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f90ab-3653-4d99-b073-8efbf8ec47cd admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:16.001575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-6e7f90ab-3653-4d99-b073-8efbf8ec47cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:16.024012 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:16.064833 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f90ab-3653-4d99-b073-8efbf8ec47cd admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:16.068917 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9097b96-63d6-48f0-a2a3-37a271e04dca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:16.069532 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9097b96-63d6-48f0-a2a3-37a271e04dca service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:16.069810 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9097b96-63d6-48f0-a2a3-37a271e04dca service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:16.070072 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9097b96-63d6-48f0-a2a3-37a271e04dca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:16.080362 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9097b96-63d6-48f0-a2a3-37a271e04dca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:16.081858 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9097b96-63d6-48f0-a2a3-37a271e04dca service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:16.093631 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1118/2243] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:34:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:34:18.335974 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b8b67-dc50-481f-b32a-5278297ee20c None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:18.341132 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-c88b8b67-dc50-481f-b32a-5278297ee20c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:18.480154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b8b67-dc50-481f-b32a-5278297ee20c None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:18.487405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5dbaf-2a58-41d4-bfe8-42cab19f250e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:18.488205 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5dbaf-2a58-41d4-bfe8-42cab19f250e service nova] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:18.488478 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5dbaf-2a58-41d4-bfe8-42cab19f250e service nova] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:18.488747 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5dbaf-2a58-41d4-bfe8-42cab19f250e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:18.490576 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-bda5dbaf-2a58-41d4-bfe8-42cab19f250e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:18.505982 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5dbaf-2a58-41d4-bfe8-42cab19f250e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:18.509398 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5dbaf-2a58-41d4-bfe8-42cab19f250e service nova] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:18.515756 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:18.523536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1126/2244] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:34:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:34:21.867311 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838d7a3-0947-4aa3-81f0-b774a382c1e9 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:21.867849 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838d7a3-0947-4aa3-81f0-b774a382c1e9 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:21.867849 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838d7a3-0947-4aa3-81f0-b774a382c1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:21.901431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-7838d7a3-0947-4aa3-81f0-b774a382c1e9 None None] MFA Rules not processed for user `0e1da9439ab44ce399e72bea608c2c29`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:21.902735 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-7838d7a3-0947-4aa3-81f0-b774a382c1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:22.000455 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1119/2245] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:34:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:34:22.000978 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5dbaf-2a58-41d4-bfe8-42cab19f250e service nova] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:22.002454 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-bda5dbaf-2a58-41d4-bfe8-42cab19f250e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:22.018448 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5dbaf-2a58-41d4-bfe8-42cab19f250e service nova] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:22.024209 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ffb54f-f8c9-4709-a50c-8fc6120cbd85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:22.024861 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ffb54f-f8c9-4709-a50c-8fc6120cbd85 service nova] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:22.025154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ffb54f-f8c9-4709-a50c-8fc6120cbd85 service nova] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:22.025428 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ffb54f-f8c9-4709-a50c-8fc6120cbd85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:22.035093 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ffb54f-f8c9-4709-a50c-8fc6120cbd85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:22.038225 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ffb54f-f8c9-4709-a50c-8fc6120cbd85 service nova] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:22.049950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1127/2246] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:34:21 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:34:23.696721 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c8d2a-6fda-4cba-90a6-d278e5c4b4ae None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:23.697282 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c8d2a-6fda-4cba-90a6-d278e5c4b4ae None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:23.697282 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c8d2a-6fda-4cba-90a6-d278e5c4b4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:23.723725 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-116c8d2a-6fda-4cba-90a6-d278e5c4b4ae None None] MFA Rules not processed for user `0e1da9439ab44ce399e72bea608c2c29`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:23.725157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-116c8d2a-6fda-4cba-90a6-d278e5c4b4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:23.747090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1120/2247] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:34:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:34:24.640275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ffb54f-f8c9-4709-a50c-8fc6120cbd85 service nova] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:24.663533 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ffb54f-f8c9-4709-a50c-8fc6120cbd85 service nova] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:24.671150 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3809dac2-a500-49c8-a810-827d56bbfa70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:24.671908 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809dac2-a500-49c8-a810-827d56bbfa70 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:24.672285 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809dac2-a500-49c8-a810-827d56bbfa70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:24.672534 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809dac2-a500-49c8-a810-827d56bbfa70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:24.677088 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-3809dac2-a500-49c8-a810-827d56bbfa70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:24.707561 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3809dac2-a500-49c8-a810-827d56bbfa70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:24.709040 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3809dac2-a500-49c8-a810-827d56bbfa70 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:24.729645 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1128/2248] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:34:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:34:24.984988 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e53f8-b43f-492b-9d67-29cfbee49733 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:24.985539 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e53f8-b43f-492b-9d67-29cfbee49733 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:24.985539 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e53f8-b43f-492b-9d67-29cfbee49733 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:24.986887 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1121/2249] 10.222.0.30 () {54 vars in 921 bytes} [Wed Nov 2 07:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:34:24.996710 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384434a-d67e-47e6-b541-5151bb9c87ae None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:24.997421 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384434a-d67e-47e6-b541-5151bb9c87ae None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:24.998344 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384434a-d67e-47e6-b541-5151bb9c87ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:25.039330 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-3384434a-d67e-47e6-b541-5151bb9c87ae None None] MFA Rules not processed for user `65d51ec74e1e4c25bcc4365eb858bac0`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:25.040677 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-3384434a-d67e-47e6-b541-5151bb9c87ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:25.068045 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1129/2250] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Nov 2 07:34:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:34:25.077533 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e53f8-b43f-492b-9d67-29cfbee49733 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:25.079733 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-c13e53f8-b43f-492b-9d67-29cfbee49733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:25.103123 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e53f8-b43f-492b-9d67-29cfbee49733 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:25.114718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720f2cd-1b87-4bed-9d1b-b86e297d490b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:25.115724 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720f2cd-1b87-4bed-9d1b-b86e297d490b service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:25.116266 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720f2cd-1b87-4bed-9d1b-b86e297d490b service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:25.116730 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720f2cd-1b87-4bed-9d1b-b86e297d490b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:25.133087 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b720f2cd-1b87-4bed-9d1b-b86e297d490b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:25.134646 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b720f2cd-1b87-4bed-9d1b-b86e297d490b service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:25.150174 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1122/2251] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:34:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:34:25.260792 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918e9d4-2f04-4d2f-bd75-97e274b8e179 None None] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:25.261406 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918e9d4-2f04-4d2f-bd75-97e274b8e179 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:25.261844 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918e9d4-2f04-4d2f-bd75-97e274b8e179 None None] PATH_INFO: `/` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:25.263574 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1130/2252] 10.222.0.30 () {54 vars in 921 bytes} [Wed Nov 2 07:34:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:34:25.274091 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0c8aa-ef3e-49ef-a750-3f9d82663929 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:25.274643 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0c8aa-ef3e-49ef-a750-3f9d82663929 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:25.275096 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0c8aa-ef3e-49ef-a750-3f9d82663929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:25.307176 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-b0c0c8aa-ef3e-49ef-a750-3f9d82663929 None None] MFA Rules not processed for user `e6495385dccc4fa48c05b1feb957846c`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:34:25.309037 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-b0c0c8aa-ef3e-49ef-a750-3f9d82663929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:25.327288 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1123/2253] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Nov 2 07:34:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:34:25.333864 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918e9d4-2f04-4d2f-bd75-97e274b8e179 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:25.350121 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918e9d4-2f04-4d2f-bd75-97e274b8e179 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:25.363393 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec592774-e9a0-45e8-abc6-9c301d496231 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:25.365113 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec592774-e9a0-45e8-abc6-9c301d496231 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:25.365576 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec592774-e9a0-45e8-abc6-9c301d496231 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:25.365943 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec592774-e9a0-45e8-abc6-9c301d496231 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:25.399804 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec592774-e9a0-45e8-abc6-9c301d496231 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:25.406168 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec592774-e9a0-45e8-abc6-9c301d496231 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:25.423314 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1131/2254] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:34:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:34:32.378808 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c0c8aa-ef3e-49ef-a750-3f9d82663929 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:34:32.393226 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c0c8aa-ef3e-49ef-a750-3f9d82663929 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:34:32.399050 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df4617-e22d-4c04-9bf2-d35cafb6986c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:34:32.399657 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df4617-e22d-4c04-9bf2-d35cafb6986c service nova] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:34:32.399935 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df4617-e22d-4c04-9bf2-d35cafb6986c service nova] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:34:32.400191 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df4617-e22d-4c04-9bf2-d35cafb6986c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:34:32.401904 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-86df4617-e22d-4c04-9bf2-d35cafb6986c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:34:32.415551 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df4617-e22d-4c04-9bf2-d35cafb6986c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:34:32.416239 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df4617-e22d-4c04-9bf2-d35cafb6986c service nova] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:34:32.427275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1124/2255] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:34:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:34:35.418273 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=22 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:34:42.422646 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=67 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:28.711550 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec592774-e9a0-45e8-abc6-9c301d496231 service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:28.720886 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec592774-e9a0-45e8-abc6-9c301d496231 service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:28.724949 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802408b-c008-43ca-ae33-25ea6791582d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:28.725547 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802408b-c008-43ca-ae33-25ea6791582d service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:28.725823 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802408b-c008-43ca-ae33-25ea6791582d service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:28.726086 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802408b-c008-43ca-ae33-25ea6791582d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:28.728404 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-6802408b-c008-43ca-ae33-25ea6791582d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:28.741790 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6802408b-c008-43ca-ae33-25ea6791582d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:28.743283 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6802408b-c008-43ca-ae33-25ea6791582d service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:28.753969 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1132/2256] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:35:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:35:31.818096 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df4617-e22d-4c04-9bf2-d35cafb6986c service nova] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:31.819669 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-86df4617-e22d-4c04-9bf2-d35cafb6986c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:31.838110 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df4617-e22d-4c04-9bf2-d35cafb6986c service nova] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:31.842884 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f7fbd-c069-4232-8540-7635e494de89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:31.843648 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f7fbd-c069-4232-8540-7635e494de89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:31.843925 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f7fbd-c069-4232-8540-7635e494de89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:31.844199 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f7fbd-c069-4232-8540-7635e494de89 admin admin] PATH_INFO: `/v3/users/be586640013949589d2a7b11a7a2f3b1` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:31.845475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f7fbd-c069-4232-8540-7635e494de89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be586640013949589d2a7b11a7a2f3b1)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:31.848507 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f7fbd-c069-4232-8540-7635e494de89 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:31.905052 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-521f7fbd-c069-4232-8540-7635e494de89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be586640013949589d2a7b11a7a2f3b1'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:31.905727 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-521f7fbd-c069-4232-8540-7635e494de89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be586640013949589d2a7b11a7a2f3b1'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:31.915575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-521f7fbd-c069-4232-8540-7635e494de89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be586640013949589d2a7b11a7a2f3b1'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:31.931107 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-521f7fbd-c069-4232-8540-7635e494de89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be586640013949589d2a7b11a7a2f3b1'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:31.940431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-521f7fbd-c069-4232-8540-7635e494de89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be586640013949589d2a7b11a7a2f3b1'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:31.946868 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1125/2257] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:35:31 2022] DELETE /identity/v3/users/be586640013949589d2a7b11a7a2f3b1 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:35:32.128601 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802408b-c008-43ca-ae33-25ea6791582d service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:32.141072 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802408b-c008-43ca-ae33-25ea6791582d service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:32.201354 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95011515-d58c-46ce-bcfa-a2ec9ca7c61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:32.201915 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95011515-d58c-46ce-bcfa-a2ec9ca7c61a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:32.202192 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95011515-d58c-46ce-bcfa-a2ec9ca7c61a admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:32.202464 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95011515-d58c-46ce-bcfa-a2ec9ca7c61a admin admin] PATH_INFO: `/v3/projects/4fc2ce1f15e34c7491c2bdcbfc26c7f0` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:32.203609 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95011515-d58c-46ce-bcfa-a2ec9ca7c61a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fc2ce1f15e34c7491c2bdcbfc26c7f0)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:32.205547 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95011515-d58c-46ce-bcfa-a2ec9ca7c61a admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:32.256405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-95011515-d58c-46ce-bcfa-a2ec9ca7c61a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fc2ce1f15e34c7491c2bdcbfc26c7f0', 'request_id': 'req-95011515-d58c-46ce-bcfa-a2ec9ca7c61a'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:32.265804 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-95011515-d58c-46ce-bcfa-a2ec9ca7c61a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fc2ce1f15e34c7491c2bdcbfc26c7f0', 'request_id': 'req-95011515-d58c-46ce-bcfa-a2ec9ca7c61a'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:32.271950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-95011515-d58c-46ce-bcfa-a2ec9ca7c61a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fc2ce1f15e34c7491c2bdcbfc26c7f0', 'request_id': 'req-95011515-d58c-46ce-bcfa-a2ec9ca7c61a'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:32.283972 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-95011515-d58c-46ce-bcfa-a2ec9ca7c61a admin admin] The token cache is being invalidate because project 4fc2ce1f15e34c7491c2bdcbfc26c7f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:35:32.284254 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-95011515-d58c-46ce-bcfa-a2ec9ca7c61a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:32.285002 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-95011515-d58c-46ce-bcfa-a2ec9ca7c61a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:32.286583 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1133/2258] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:35:32 2022] DELETE /identity/v3/projects/4fc2ce1f15e34c7491c2bdcbfc26c7f0 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:32.292018 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed37dc-672c-4c27-b300-d2056e287da8 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:32.292303 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed37dc-672c-4c27-b300-d2056e287da8 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:32.292571 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed37dc-672c-4c27-b300-d2056e287da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:32.312729 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-40ed37dc-672c-4c27-b300-d2056e287da8 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:32.313926 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-40ed37dc-672c-4c27-b300-d2056e287da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:32.377458 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1126/2259] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:35:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:32.385878 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61889d33-8f8b-45ce-b591-9b1079e72e43 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:32.386163 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61889d33-8f8b-45ce-b591-9b1079e72e43 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:32.386434 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61889d33-8f8b-45ce-b591-9b1079e72e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:32.408040 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-61889d33-8f8b-45ce-b591-9b1079e72e43 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:32.409258 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-61889d33-8f8b-45ce-b591-9b1079e72e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:32.418976 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1134/2260] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:35:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:32.424124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ed37dc-672c-4c27-b300-d2056e287da8 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:32.438494 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ed37dc-672c-4c27-b300-d2056e287da8 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:32.445778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c2bf8-1ceb-4c72-9a4e-03bd628164b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:32.446381 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c2bf8-1ceb-4c72-9a4e-03bd628164b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:32.446648 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c2bf8-1ceb-4c72-9a4e-03bd628164b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:32.446948 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c2bf8-1ceb-4c72-9a4e-03bd628164b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:32.447791 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c2bf8-1ceb-4c72-9a4e-03bd628164b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:35:32.448188 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c2bf8-1ceb-4c72-9a4e-03bd628164b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:32.450551 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c2bf8-1ceb-4c72-9a4e-03bd628164b6 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:32.463327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1127/2261] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:35:32.467906 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61889d33-8f8b-45ce-b591-9b1079e72e43 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:32.477376 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61889d33-8f8b-45ce-b591-9b1079e72e43 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:32.482296 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc95354-9969-448f-aad8-7a79fee9217c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:32.482924 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc95354-9969-448f-aad8-7a79fee9217c admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:32.483206 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc95354-9969-448f-aad8-7a79fee9217c admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:32.483461 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc95354-9969-448f-aad8-7a79fee9217c admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:32.484086 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc95354-9969-448f-aad8-7a79fee9217c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:32.485999 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc95354-9969-448f-aad8-7a79fee9217c admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:32.515071 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1135/2262] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:35:32 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:35:32.519277 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c2bf8-1ceb-4c72-9a4e-03bd628164b6 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:32.529157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c2bf8-1ceb-4c72-9a4e-03bd628164b6 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:32.592299 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c19467-7cc3-489c-8c51-13e408bd03b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:32.592923 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c19467-7cc3-489c-8c51-13e408bd03b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:32.593197 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c19467-7cc3-489c-8c51-13e408bd03b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:32.593497 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c19467-7cc3-489c-8c51-13e408bd03b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:32.594117 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c19467-7cc3-489c-8c51-13e408bd03b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:32.596060 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c19467-7cc3-489c-8c51-13e408bd03b7 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:32.604281 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:32.624129 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1128/2263] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:35:32 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:35:32.628563 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc95354-9969-448f-aad8-7a79fee9217c admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:32.639816 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc95354-9969-448f-aad8-7a79fee9217c admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:32.645733 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a996031-a7f2-4806-81c4-ea045be0ef04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:32.646330 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a996031-a7f2-4806-81c4-ea045be0ef04 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:32.646602 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a996031-a7f2-4806-81c4-ea045be0ef04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:32.646918 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a996031-a7f2-4806-81c4-ea045be0ef04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:32.647553 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a996031-a7f2-4806-81c4-ea045be0ef04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:32.649144 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a996031-a7f2-4806-81c4-ea045be0ef04 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:32.663604 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1136/2264] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:35:32.668100 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c19467-7cc3-489c-8c51-13e408bd03b7 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:32.679173 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c19467-7cc3-489c-8c51-13e408bd03b7 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:32.684529 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b6273-799f-43ed-b6b7-be7cdea85508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:32.685271 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b6273-799f-43ed-b6b7-be7cdea85508 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:32.685550 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b6273-799f-43ed-b6b7-be7cdea85508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:32.685821 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b6273-799f-43ed-b6b7-be7cdea85508 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:32.686468 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b6273-799f-43ed-b6b7-be7cdea85508 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:32.688113 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b6273-799f-43ed-b6b7-be7cdea85508 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:32.698953 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1129/2265] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:35:32.704727 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a996031-a7f2-4806-81c4-ea045be0ef04 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:32.715243 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a996031-a7f2-4806-81c4-ea045be0ef04 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:32.720751 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75869377-aa9c-4935-bbc8-f785c782ea12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:32.721404 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75869377-aa9c-4935-bbc8-f785c782ea12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:32.721701 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75869377-aa9c-4935-bbc8-f785c782ea12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:32.721968 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75869377-aa9c-4935-bbc8-f785c782ea12 admin admin] PATH_INFO: `/v3/projects/b67eb8161ea34bb7b7e9d3bbc88eb6b5/users/47c4f9e4e0cd437d9715e6e03e47fd4f/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:32.736894 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75869377-aa9c-4935-bbc8-f785c782ea12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b67eb8161ea34bb7b7e9d3bbc88eb6b5, user_id=47c4f9e4e0cd437d9715e6e03e47fd4f, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:32.739635 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75869377-aa9c-4935-bbc8-f785c782ea12 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:32.755349 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1137/2266] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:35:32 2022] PUT /identity/v3/projects/b67eb8161ea34bb7b7e9d3bbc88eb6b5/users/47c4f9e4e0cd437d9715e6e03e47fd4f/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:32.768353 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c613167-5897-4191-8284-1c50fba4029b None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:32.768719 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c613167-5897-4191-8284-1c50fba4029b None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:32.769042 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c613167-5897-4191-8284-1c50fba4029b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:32.800613 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-5c613167-5897-4191-8284-1c50fba4029b None None] MFA Rules not processed for user `47c4f9e4e0cd437d9715e6e03e47fd4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:32.801821 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-5c613167-5897-4191-8284-1c50fba4029b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:32.868509 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1130/2267] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:35:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:32.880352 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75869377-aa9c-4935-bbc8-f785c782ea12 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:32.882271 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-75869377-aa9c-4935-bbc8-f785c782ea12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:32.897220 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:32.960075 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75869377-aa9c-4935-bbc8-f785c782ea12 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:32.965692 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296bd478-f209-41d7-ad45-d6719b0fb88c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:32.966230 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bd478-f209-41d7-ad45-d6719b0fb88c service nova] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:32.966509 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bd478-f209-41d7-ad45-d6719b0fb88c service nova] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:32.966803 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bd478-f209-41d7-ad45-d6719b0fb88c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:32.975123 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296bd478-f209-41d7-ad45-d6719b0fb88c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:32.976725 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296bd478-f209-41d7-ad45-d6719b0fb88c service nova] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:32.986037 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1138/2268] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:35:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:35:35.718374 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c613167-5897-4191-8284-1c50fba4029b None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:35.781761 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c613167-5897-4191-8284-1c50fba4029b None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:35.786625 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204ee031-be74-447e-aced-713e2a4da3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:35.787280 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ee031-be74-447e-aced-713e2a4da3ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:35.787553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ee031-be74-447e-aced-713e2a4da3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:35.787817 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ee031-be74-447e-aced-713e2a4da3ec admin admin] PATH_INFO: `/v3/users/47c4f9e4e0cd437d9715e6e03e47fd4f` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:35.788950 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204ee031-be74-447e-aced-713e2a4da3ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=47c4f9e4e0cd437d9715e6e03e47fd4f)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:35.790902 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204ee031-be74-447e-aced-713e2a4da3ec admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:35.843595 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-204ee031-be74-447e-aced-713e2a4da3ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47c4f9e4e0cd437d9715e6e03e47fd4f'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:35.844206 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-204ee031-be74-447e-aced-713e2a4da3ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47c4f9e4e0cd437d9715e6e03e47fd4f'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:35.851185 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-204ee031-be74-447e-aced-713e2a4da3ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47c4f9e4e0cd437d9715e6e03e47fd4f'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:35.861985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:35.864251 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-204ee031-be74-447e-aced-713e2a4da3ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47c4f9e4e0cd437d9715e6e03e47fd4f'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:35.873943 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-204ee031-be74-447e-aced-713e2a4da3ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47c4f9e4e0cd437d9715e6e03e47fd4f'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:35.880723 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1131/2269] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:35:35 2022] DELETE /identity/v3/users/47c4f9e4e0cd437d9715e6e03e47fd4f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:35.885374 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ee1d6-ad99-4829-beeb-4b1989f3c51f None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:35.885861 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ee1d6-ad99-4829-beeb-4b1989f3c51f None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:35.885987 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ee1d6-ad99-4829-beeb-4b1989f3c51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:35.905836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-076ee1d6-ad99-4829-beeb-4b1989f3c51f None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:35.907086 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-076ee1d6-ad99-4829-beeb-4b1989f3c51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:35.969231 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1139/2270] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:35:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:35.979263 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204ee031-be74-447e-aced-713e2a4da3ec admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:35.980727 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-204ee031-be74-447e-aced-713e2a4da3ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:36.042228 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204ee031-be74-447e-aced-713e2a4da3ec admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:36.046876 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e2f2e-1b89-4d78-853d-9ec00a0bdc45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:36.047491 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e2f2e-1b89-4d78-853d-9ec00a0bdc45 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:36.047761 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e2f2e-1b89-4d78-853d-9ec00a0bdc45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:36.048035 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e2f2e-1b89-4d78-853d-9ec00a0bdc45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:36.058228 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e2f2e-1b89-4d78-853d-9ec00a0bdc45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:36.059721 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e2f2e-1b89-4d78-853d-9ec00a0bdc45 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:36.070083 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1132/2271] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:35:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:35:36.696530 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076ee1d6-ad99-4829-beeb-4b1989f3c51f None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:36.698104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-076ee1d6-ad99-4829-beeb-4b1989f3c51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:36.712456 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076ee1d6-ad99-4829-beeb-4b1989f3c51f None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:36.717721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7405e6-c8d8-45c6-adc7-ec3ef7978d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:36.718346 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7405e6-c8d8-45c6-adc7-ec3ef7978d74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:36.719258 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7405e6-c8d8-45c6-adc7-ec3ef7978d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:36.719544 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7405e6-c8d8-45c6-adc7-ec3ef7978d74 admin admin] PATH_INFO: `/v3/projects/b67eb8161ea34bb7b7e9d3bbc88eb6b5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:36.720908 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7405e6-c8d8-45c6-adc7-ec3ef7978d74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b67eb8161ea34bb7b7e9d3bbc88eb6b5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:36.723157 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7405e6-c8d8-45c6-adc7-ec3ef7978d74 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:36.777172 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-8a7405e6-c8d8-45c6-adc7-ec3ef7978d74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b67eb8161ea34bb7b7e9d3bbc88eb6b5', 'request_id': 'req-8a7405e6-c8d8-45c6-adc7-ec3ef7978d74'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:36.786497 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-8a7405e6-c8d8-45c6-adc7-ec3ef7978d74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b67eb8161ea34bb7b7e9d3bbc88eb6b5', 'request_id': 'req-8a7405e6-c8d8-45c6-adc7-ec3ef7978d74'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:36.793266 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-8a7405e6-c8d8-45c6-adc7-ec3ef7978d74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b67eb8161ea34bb7b7e9d3bbc88eb6b5', 'request_id': 'req-8a7405e6-c8d8-45c6-adc7-ec3ef7978d74'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:36.805416 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-8a7405e6-c8d8-45c6-adc7-ec3ef7978d74 admin admin] The token cache is being invalidate because project b67eb8161ea34bb7b7e9d3bbc88eb6b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:35:36.805702 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-8a7405e6-c8d8-45c6-adc7-ec3ef7978d74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:36.806336 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-8a7405e6-c8d8-45c6-adc7-ec3ef7978d74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:36.807977 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1140/2272] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:35:36 2022] DELETE /identity/v3/projects/b67eb8161ea34bb7b7e9d3bbc88eb6b5 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:36.813446 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b02c44-087b-470c-aef5-ac71dd183ec5 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:36.813963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b02c44-087b-470c-aef5-ac71dd183ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:36.813963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b02c44-087b-470c-aef5-ac71dd183ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:36.834967 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-e2b02c44-087b-470c-aef5-ac71dd183ec5 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:36.836165 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-e2b02c44-087b-470c-aef5-ac71dd183ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:36.898686 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1133/2273] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:35:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:36.907245 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2c9fa-e10a-4009-b9d4-106e816f8ef8 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:36.907744 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2c9fa-e10a-4009-b9d4-106e816f8ef8 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:36.907744 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2c9fa-e10a-4009-b9d4-106e816f8ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:36.928457 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-dcb2c9fa-e10a-4009-b9d4-106e816f8ef8 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:36.929684 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-dcb2c9fa-e10a-4009-b9d4-106e816f8ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:36.938240 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1141/2274] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:35:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:36.942673 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b02c44-087b-470c-aef5-ac71dd183ec5 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:36.952749 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b02c44-087b-470c-aef5-ac71dd183ec5 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:36.958157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b95f7c-821a-43fa-807a-56e2d3baea28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:36.958786 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b95f7c-821a-43fa-807a-56e2d3baea28 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:36.959062 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b95f7c-821a-43fa-807a-56e2d3baea28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:36.959330 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b95f7c-821a-43fa-807a-56e2d3baea28 admin admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:36.959970 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b95f7c-821a-43fa-807a-56e2d3baea28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:35:36.960268 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b95f7c-821a-43fa-807a-56e2d3baea28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:36.961866 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b95f7c-821a-43fa-807a-56e2d3baea28 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:36.973992 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1134/2275] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:35:36.978560 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2c9fa-e10a-4009-b9d4-106e816f8ef8 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:36.988453 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2c9fa-e10a-4009-b9d4-106e816f8ef8 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:36.994762 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f314c5a8-4cd8-465d-817d-6e70a8814179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:36.995367 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314c5a8-4cd8-465d-817d-6e70a8814179 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:36.995645 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314c5a8-4cd8-465d-817d-6e70a8814179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:36.995910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314c5a8-4cd8-465d-817d-6e70a8814179 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:36.996548 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f314c5a8-4cd8-465d-817d-6e70a8814179 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:36.998470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f314c5a8-4cd8-465d-817d-6e70a8814179 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:37.014274 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:37.029444 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1142/2276] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:35:36 2022] POST /identity/v3/projects => generated 363 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:35:37.034229 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b95f7c-821a-43fa-807a-56e2d3baea28 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:37.044539 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b95f7c-821a-43fa-807a-56e2d3baea28 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:37.107598 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c18166-16f2-49cf-8c73-e47336a71089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:37.108237 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c18166-16f2-49cf-8c73-e47336a71089 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:37.108510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c18166-16f2-49cf-8c73-e47336a71089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:37.108760 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c18166-16f2-49cf-8c73-e47336a71089 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:37.109398 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c18166-16f2-49cf-8c73-e47336a71089 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:37.111364 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c18166-16f2-49cf-8c73-e47336a71089 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:37.135882 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1135/2277] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:35:37 2022] POST /identity/v3/users => generated 282 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:35:37.140681 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f314c5a8-4cd8-465d-817d-6e70a8814179 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:37.151100 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f314c5a8-4cd8-465d-817d-6e70a8814179 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:37.155999 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5c079-bf54-499e-821f-7278611d294f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:37.156593 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5c079-bf54-499e-821f-7278611d294f admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:37.156864 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5c079-bf54-499e-821f-7278611d294f admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:37.157126 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5c079-bf54-499e-821f-7278611d294f admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:37.157780 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de5c079-bf54-499e-821f-7278611d294f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:37.159421 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de5c079-bf54-499e-821f-7278611d294f admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:37.169281 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1143/2278] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:35:37.173478 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c18166-16f2-49cf-8c73-e47336a71089 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:37.183915 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c18166-16f2-49cf-8c73-e47336a71089 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:37.189222 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982f4a7-fa01-42e9-8fc5-87111d181a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:37.189818 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982f4a7-fa01-42e9-8fc5-87111d181a3a admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:37.190090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982f4a7-fa01-42e9-8fc5-87111d181a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:37.190387 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982f4a7-fa01-42e9-8fc5-87111d181a3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:37.191058 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6982f4a7-fa01-42e9-8fc5-87111d181a3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:37.192717 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6982f4a7-fa01-42e9-8fc5-87111d181a3a admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:37.193982 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:37.206845 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1136/2279] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:35:37.210912 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5c079-bf54-499e-821f-7278611d294f admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:37.221770 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5c079-bf54-499e-821f-7278611d294f admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:37.228112 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b82ad5-da44-4407-88d4-4e9b4e92ba30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:37.228771 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b82ad5-da44-4407-88d4-4e9b4e92ba30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:37.229048 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b82ad5-da44-4407-88d4-4e9b4e92ba30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:37.229318 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b82ad5-da44-4407-88d4-4e9b4e92ba30 admin admin] PATH_INFO: `/v3/projects/e7df132e09ad45b8ae175e75e9e9a6ec/users/93c10e481f264ee38f5a5af43111e85b/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:37.248701 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b82ad5-da44-4407-88d4-4e9b4e92ba30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7df132e09ad45b8ae175e75e9e9a6ec, user_id=93c10e481f264ee38f5a5af43111e85b, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:37.251554 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b82ad5-da44-4407-88d4-4e9b4e92ba30 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:37.268527 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1144/2280] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:35:37 2022] PUT /identity/v3/projects/e7df132e09ad45b8ae175e75e9e9a6ec/users/93c10e481f264ee38f5a5af43111e85b/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:37.279439 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f182c-c4c3-44b5-bc7b-cd1a9337073f None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:37.279814 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f182c-c4c3-44b5-bc7b-cd1a9337073f None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:37.280099 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f182c-c4c3-44b5-bc7b-cd1a9337073f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:37.309524 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-d66f182c-c4c3-44b5-bc7b-cd1a9337073f None None] MFA Rules not processed for user `93c10e481f264ee38f5a5af43111e85b`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:37.310753 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-d66f182c-c4c3-44b5-bc7b-cd1a9337073f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:37.380357 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1137/2281] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:35:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:37.385244 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b82ad5-da44-4407-88d4-4e9b4e92ba30 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:37.455722 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b82ad5-da44-4407-88d4-4e9b4e92ba30 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:37.461286 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e110a03-5010-4e51-91c7-a1f5cdfa7dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:37.461818 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e110a03-5010-4e51-91c7-a1f5cdfa7dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:37.462090 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e110a03-5010-4e51-91c7-a1f5cdfa7dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:37.462351 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e110a03-5010-4e51-91c7-a1f5cdfa7dc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:37.463004 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e110a03-5010-4e51-91c7-a1f5cdfa7dc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:37.464902 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e110a03-5010-4e51-91c7-a1f5cdfa7dc0 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:37.495981 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1145/2282] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:35:37 2022] POST /identity/v3/projects => generated 363 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:35:37.501088 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f182c-c4c3-44b5-bc7b-cd1a9337073f None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:37.517016 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f182c-c4c3-44b5-bc7b-cd1a9337073f None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:37.576047 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e793ae-7e09-434c-a0c9-4e4fea8fff4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:37.576733 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e793ae-7e09-434c-a0c9-4e4fea8fff4e admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:37.577019 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e793ae-7e09-434c-a0c9-4e4fea8fff4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:37.577365 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e793ae-7e09-434c-a0c9-4e4fea8fff4e admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:37.577983 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e793ae-7e09-434c-a0c9-4e4fea8fff4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:37.579931 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e793ae-7e09-434c-a0c9-4e4fea8fff4e admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:37.602580 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1138/2283] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:35:37 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:35:37.606679 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e110a03-5010-4e51-91c7-a1f5cdfa7dc0 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:37.619571 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e110a03-5010-4e51-91c7-a1f5cdfa7dc0 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:37.624351 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a05878-370d-44af-8766-a1af7959a315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:37.624951 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a05878-370d-44af-8766-a1af7959a315 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:37.625214 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a05878-370d-44af-8766-a1af7959a315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:37.625478 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a05878-370d-44af-8766-a1af7959a315 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:37.626111 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a05878-370d-44af-8766-a1af7959a315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:37.627734 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a05878-370d-44af-8766-a1af7959a315 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:37.638191 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1146/2284] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:35:37.642773 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e793ae-7e09-434c-a0c9-4e4fea8fff4e admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:37.653032 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e793ae-7e09-434c-a0c9-4e4fea8fff4e admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:37.658595 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b71c52-ac41-459c-9737-3e2f2b3feb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:37.659267 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b71c52-ac41-459c-9737-3e2f2b3feb59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:37.659539 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b71c52-ac41-459c-9737-3e2f2b3feb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:37.659804 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b71c52-ac41-459c-9737-3e2f2b3feb59 admin admin] PATH_INFO: `/v3/projects/7f8caea105b34dfcbc1cd31aa9b1148b/users/3b48a93759b4407d8d6a43573cc7f818/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:37.674904 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b71c52-ac41-459c-9737-3e2f2b3feb59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f8caea105b34dfcbc1cd31aa9b1148b, user_id=3b48a93759b4407d8d6a43573cc7f818, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:37.677603 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b71c52-ac41-459c-9737-3e2f2b3feb59 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:37.693459 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1139/2285] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:35:37 2022] PUT /identity/v3/projects/7f8caea105b34dfcbc1cd31aa9b1148b/users/3b48a93759b4407d8d6a43573cc7f818/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:37.697573 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a05878-370d-44af-8766-a1af7959a315 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:37.708119 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a05878-370d-44af-8766-a1af7959a315 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:37.767897 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b57a6d-c345-4c45-a44a-fe02e44d139c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:37.768517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b57a6d-c345-4c45-a44a-fe02e44d139c admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:37.768793 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b57a6d-c345-4c45-a44a-fe02e44d139c admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:37.769061 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b57a6d-c345-4c45-a44a-fe02e44d139c admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:37.769709 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b57a6d-c345-4c45-a44a-fe02e44d139c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:37.771342 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b57a6d-c345-4c45-a44a-fe02e44d139c admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:37.781424 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1147/2286] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:35:37.786045 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b71c52-ac41-459c-9737-3e2f2b3feb59 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:37.796550 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b71c52-ac41-459c-9737-3e2f2b3feb59 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:37.801759 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e35df5-a9a3-4235-b09d-7923001884fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:37.802440 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e35df5-a9a3-4235-b09d-7923001884fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:37.802791 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e35df5-a9a3-4235-b09d-7923001884fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:37.803034 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e35df5-a9a3-4235-b09d-7923001884fe admin admin] PATH_INFO: `/v3/projects/7f8caea105b34dfcbc1cd31aa9b1148b/users/3b48a93759b4407d8d6a43573cc7f818/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:37.804973 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e35df5-a9a3-4235-b09d-7923001884fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f8caea105b34dfcbc1cd31aa9b1148b, user_id=3b48a93759b4407d8d6a43573cc7f818, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:37.807668 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e35df5-a9a3-4235-b09d-7923001884fe admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:37.818404 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1140/2287] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:35:37 2022] PUT /identity/v3/projects/7f8caea105b34dfcbc1cd31aa9b1148b/users/3b48a93759b4407d8d6a43573cc7f818/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:35:37.830534 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a558f9e-868a-4605-969c-7dab0ed95838 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:37.830903 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a558f9e-868a-4605-969c-7dab0ed95838 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:37.831097 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a558f9e-868a-4605-969c-7dab0ed95838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:37.860776 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-8a558f9e-868a-4605-969c-7dab0ed95838 None None] MFA Rules not processed for user `3b48a93759b4407d8d6a43573cc7f818`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:37.861987 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-8a558f9e-868a-4605-969c-7dab0ed95838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:37.879528 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:37.938019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1148/2288] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:35:37 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:37.950183 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e35df5-a9a3-4235-b09d-7923001884fe admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:37.952530 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-f8e35df5-a9a3-4235-b09d-7923001884fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:37.999073 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:38.016885 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e35df5-a9a3-4235-b09d-7923001884fe admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:38.020955 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebbdd4-9911-46c6-88f9-5cf94e1ef149 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:38.021603 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebbdd4-9911-46c6-88f9-5cf94e1ef149 service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:38.021871 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebbdd4-9911-46c6-88f9-5cf94e1ef149 service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:38.022139 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebbdd4-9911-46c6-88f9-5cf94e1ef149 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:38.032174 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ebbdd4-9911-46c6-88f9-5cf94e1ef149 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:38.033612 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ebbdd4-9911-46c6-88f9-5cf94e1ef149 service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:38.043273 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1141/2289] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:35:37 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:35:38.520030 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a558f9e-868a-4605-969c-7dab0ed95838 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:38.585861 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a558f9e-868a-4605-969c-7dab0ed95838 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:38.591850 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6b48a-5bbd-47c3-a5ec-93b2a575becb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:38.592489 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6b48a-5bbd-47c3-a5ec-93b2a575becb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:38.592767 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6b48a-5bbd-47c3-a5ec-93b2a575becb admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:38.593151 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6b48a-5bbd-47c3-a5ec-93b2a575becb admin admin] PATH_INFO: `/v3/users/93c10e481f264ee38f5a5af43111e85b` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:38.594427 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc6b48a-5bbd-47c3-a5ec-93b2a575becb admin admin] RBAC: Authorizing `identity:delete_user(user_id=93c10e481f264ee38f5a5af43111e85b)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:38.596447 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc6b48a-5bbd-47c3-a5ec-93b2a575becb admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:38.647335 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-7bc6b48a-5bbd-47c3-a5ec-93b2a575becb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93c10e481f264ee38f5a5af43111e85b'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:38.654062 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-7bc6b48a-5bbd-47c3-a5ec-93b2a575becb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93c10e481f264ee38f5a5af43111e85b'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:38.654760 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-7bc6b48a-5bbd-47c3-a5ec-93b2a575becb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93c10e481f264ee38f5a5af43111e85b'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:38.667660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-7bc6b48a-5bbd-47c3-a5ec-93b2a575becb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93c10e481f264ee38f5a5af43111e85b'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:38.677549 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-7bc6b48a-5bbd-47c3-a5ec-93b2a575becb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93c10e481f264ee38f5a5af43111e85b'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:38.684826 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1149/2290] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:35:38 2022] DELETE /identity/v3/users/93c10e481f264ee38f5a5af43111e85b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:38.690003 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263a929-87f5-4d8d-97f1-92a91d3c2238 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:38.690284 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263a929-87f5-4d8d-97f1-92a91d3c2238 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:38.690554 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263a929-87f5-4d8d-97f1-92a91d3c2238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:38.709517 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-8263a929-87f5-4d8d-97f1-92a91d3c2238 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:38.710698 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-8263a929-87f5-4d8d-97f1-92a91d3c2238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:38.773397 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1142/2291] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:35:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:38.783286 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6b48a-5bbd-47c3-a5ec-93b2a575becb admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:38.784749 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-7bc6b48a-5bbd-47c3-a5ec-93b2a575becb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:38.819642 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:38.842608 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6b48a-5bbd-47c3-a5ec-93b2a575becb admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:38.846576 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febdbb61-f85a-4bc2-bbed-4cc6638631ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:38.847227 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febdbb61-f85a-4bc2-bbed-4cc6638631ce service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:38.847501 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febdbb61-f85a-4bc2-bbed-4cc6638631ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:38.847769 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febdbb61-f85a-4bc2-bbed-4cc6638631ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:38.856946 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febdbb61-f85a-4bc2-bbed-4cc6638631ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:38.858380 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febdbb61-f85a-4bc2-bbed-4cc6638631ce service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:38.867813 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1150/2292] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:35:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:35:39.156387 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8263a929-87f5-4d8d-97f1-92a91d3c2238 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:39.158484 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-8263a929-87f5-4d8d-97f1-92a91d3c2238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:39.171459 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8263a929-87f5-4d8d-97f1-92a91d3c2238 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:39.176205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28520adb-84b9-45e7-b250-728949214e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:39.176846 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28520adb-84b9-45e7-b250-728949214e51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:39.177121 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28520adb-84b9-45e7-b250-728949214e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:39.177556 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28520adb-84b9-45e7-b250-728949214e51 admin admin] PATH_INFO: `/v3/projects/e7df132e09ad45b8ae175e75e9e9a6ec` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:39.178783 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28520adb-84b9-45e7-b250-728949214e51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7df132e09ad45b8ae175e75e9e9a6ec)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:39.180749 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28520adb-84b9-45e7-b250-728949214e51 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:39.231318 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-28520adb-84b9-45e7-b250-728949214e51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7df132e09ad45b8ae175e75e9e9a6ec', 'request_id': 'req-28520adb-84b9-45e7-b250-728949214e51'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:39.237988 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-28520adb-84b9-45e7-b250-728949214e51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7df132e09ad45b8ae175e75e9e9a6ec', 'request_id': 'req-28520adb-84b9-45e7-b250-728949214e51'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:39.243717 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-28520adb-84b9-45e7-b250-728949214e51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7df132e09ad45b8ae175e75e9e9a6ec', 'request_id': 'req-28520adb-84b9-45e7-b250-728949214e51'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:39.259111 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-28520adb-84b9-45e7-b250-728949214e51 admin admin] The token cache is being invalidate because project e7df132e09ad45b8ae175e75e9e9a6ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:35:39.259407 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-28520adb-84b9-45e7-b250-728949214e51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:39.260130 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-28520adb-84b9-45e7-b250-728949214e51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:39.261625 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1143/2293] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:35:39 2022] DELETE /identity/v3/projects/e7df132e09ad45b8ae175e75e9e9a6ec => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:39.265875 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febdbb61-f85a-4bc2-bbed-4cc6638631ce service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:39.275829 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febdbb61-f85a-4bc2-bbed-4cc6638631ce service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:39.333450 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a8806a-4879-43de-a284-732870ed3c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:39.334069 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8806a-4879-43de-a284-732870ed3c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:39.334356 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8806a-4879-43de-a284-732870ed3c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:39.334623 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8806a-4879-43de-a284-732870ed3c6f admin admin] PATH_INFO: `/v3/users/3b48a93759b4407d8d6a43573cc7f818` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:39.335870 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a8806a-4879-43de-a284-732870ed3c6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b48a93759b4407d8d6a43573cc7f818)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:39.337789 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a8806a-4879-43de-a284-732870ed3c6f admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:39.389495 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-22a8806a-4879-43de-a284-732870ed3c6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b48a93759b4407d8d6a43573cc7f818'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:39.396570 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-22a8806a-4879-43de-a284-732870ed3c6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b48a93759b4407d8d6a43573cc7f818'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:39.397236 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-22a8806a-4879-43de-a284-732870ed3c6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b48a93759b4407d8d6a43573cc7f818'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:39.409403 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-22a8806a-4879-43de-a284-732870ed3c6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b48a93759b4407d8d6a43573cc7f818'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:39.418145 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-22a8806a-4879-43de-a284-732870ed3c6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b48a93759b4407d8d6a43573cc7f818'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:39.425491 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1151/2294] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:35:39 2022] DELETE /identity/v3/users/3b48a93759b4407d8d6a43573cc7f818 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:39.715490 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28520adb-84b9-45e7-b250-728949214e51 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:39.717106 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-28520adb-84b9-45e7-b250-728949214e51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:39.730980 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28520adb-84b9-45e7-b250-728949214e51 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:39.791551 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03accb42-e03f-468c-a698-574d659a0ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:39.792183 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03accb42-e03f-468c-a698-574d659a0ba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:39.792460 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03accb42-e03f-468c-a698-574d659a0ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:39.792725 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03accb42-e03f-468c-a698-574d659a0ba5 admin admin] PATH_INFO: `/v3/projects/7f8caea105b34dfcbc1cd31aa9b1148b` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:39.793853 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03accb42-e03f-468c-a698-574d659a0ba5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f8caea105b34dfcbc1cd31aa9b1148b)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:39.795828 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03accb42-e03f-468c-a698-574d659a0ba5 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:39.804502 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:39.854112 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-03accb42-e03f-468c-a698-574d659a0ba5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f8caea105b34dfcbc1cd31aa9b1148b', 'request_id': 'req-03accb42-e03f-468c-a698-574d659a0ba5'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:39.860613 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-03accb42-e03f-468c-a698-574d659a0ba5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f8caea105b34dfcbc1cd31aa9b1148b', 'request_id': 'req-03accb42-e03f-468c-a698-574d659a0ba5'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:39.866525 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-03accb42-e03f-468c-a698-574d659a0ba5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f8caea105b34dfcbc1cd31aa9b1148b', 'request_id': 'req-03accb42-e03f-468c-a698-574d659a0ba5'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:39.880727 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-03accb42-e03f-468c-a698-574d659a0ba5 admin admin] The token cache is being invalidate because project 7f8caea105b34dfcbc1cd31aa9b1148b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:35:39.881012 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-03accb42-e03f-468c-a698-574d659a0ba5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:39.881611 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-03accb42-e03f-468c-a698-574d659a0ba5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:39.883155 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1144/2295] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:35:39 2022] DELETE /identity/v3/projects/7f8caea105b34dfcbc1cd31aa9b1148b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:39.888121 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a96a46-95b9-4376-bb4f-4907d95171fd None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:39.888406 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a96a46-95b9-4376-bb4f-4907d95171fd None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:39.888679 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a96a46-95b9-4376-bb4f-4907d95171fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:39.908336 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-d8a96a46-95b9-4376-bb4f-4907d95171fd None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:39.909558 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-d8a96a46-95b9-4376-bb4f-4907d95171fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:39.962562 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:39.980218 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1152/2296] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:35:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:39.988788 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dc72cd-ff27-4104-a578-435fd16900fb None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:39.989076 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dc72cd-ff27-4104-a578-435fd16900fb None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:39.989335 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dc72cd-ff27-4104-a578-435fd16900fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:40.011009 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-84dc72cd-ff27-4104-a578-435fd16900fb None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:40.012238 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-84dc72cd-ff27-4104-a578-435fd16900fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:40.021743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1145/2297] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:35:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:35:40.026224 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a96a46-95b9-4376-bb4f-4907d95171fd None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:40.036575 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a96a46-95b9-4376-bb4f-4907d95171fd None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:40.041574 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05a9e33-e55e-4614-b029-d6d3d94670c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:40.042170 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a9e33-e55e-4614-b029-d6d3d94670c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:40.042440 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a9e33-e55e-4614-b029-d6d3d94670c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:40.042682 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a9e33-e55e-4614-b029-d6d3d94670c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:40.043367 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05a9e33-e55e-4614-b029-d6d3d94670c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:35:40.043660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05a9e33-e55e-4614-b029-d6d3d94670c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:40.045245 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05a9e33-e55e-4614-b029-d6d3d94670c5 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:40.057388 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1153/2298] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:35:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:35:40.061993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dc72cd-ff27-4104-a578-435fd16900fb None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:40.071863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dc72cd-ff27-4104-a578-435fd16900fb None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:40.077102 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f5a95-279a-425b-9e86-b91c8759a265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:40.077763 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f5a95-279a-425b-9e86-b91c8759a265 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:40.078047 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f5a95-279a-425b-9e86-b91c8759a265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:40.078318 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f5a95-279a-425b-9e86-b91c8759a265 admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:40.079020 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16f5a95-279a-425b-9e86-b91c8759a265 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:40.081176 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16f5a95-279a-425b-9e86-b91c8759a265 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:40.114149 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1146/2299] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:35:40 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:35:40.121485 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05a9e33-e55e-4614-b029-d6d3d94670c5 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:40.132749 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05a9e33-e55e-4614-b029-d6d3d94670c5 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:40.192856 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c0a93d-004e-4cbb-83e1-9c473942c84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:40.193475 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0a93d-004e-4cbb-83e1-9c473942c84c admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:40.193750 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0a93d-004e-4cbb-83e1-9c473942c84c admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:40.194018 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0a93d-004e-4cbb-83e1-9c473942c84c admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:40.194668 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c0a93d-004e-4cbb-83e1-9c473942c84c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:40.196623 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c0a93d-004e-4cbb-83e1-9c473942c84c admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:40.222818 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1154/2300] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:35:40 2022] POST /identity/v3/users => generated 285 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:35:40.223968 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f5a95-279a-425b-9e86-b91c8759a265 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:40.234116 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f5a95-279a-425b-9e86-b91c8759a265 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:40.239130 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a6aa83-9d9f-4b93-a2a7-3af5fe097e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:40.239730 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6aa83-9d9f-4b93-a2a7-3af5fe097e1d admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:40.240004 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6aa83-9d9f-4b93-a2a7-3af5fe097e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:40.240267 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6aa83-9d9f-4b93-a2a7-3af5fe097e1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:40.240902 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a6aa83-9d9f-4b93-a2a7-3af5fe097e1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:40.242478 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a6aa83-9d9f-4b93-a2a7-3af5fe097e1d admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:40.253717 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1147/2301] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:35:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:35:40.257796 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c0a93d-004e-4cbb-83e1-9c473942c84c admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:40.268679 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c0a93d-004e-4cbb-83e1-9c473942c84c admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:40.274172 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a6a5e-e50a-4ddb-8368-b27fcd32a59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:40.274838 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a6a5e-e50a-4ddb-8368-b27fcd32a59e admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:40.275087 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a6a5e-e50a-4ddb-8368-b27fcd32a59e admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:40.275375 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a6a5e-e50a-4ddb-8368-b27fcd32a59e admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:40.276007 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a6a5e-e50a-4ddb-8368-b27fcd32a59e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:40.277631 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a6a5e-e50a-4ddb-8368-b27fcd32a59e admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:40.289172 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1155/2302] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:35:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:35:40.293693 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a6aa83-9d9f-4b93-a2a7-3af5fe097e1d admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:40.304222 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a6aa83-9d9f-4b93-a2a7-3af5fe097e1d admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:40.309887 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e52fb-a3c3-431e-a288-b6c336b7d3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:40.310538 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e52fb-a3c3-431e-a288-b6c336b7d3de admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:40.310847 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e52fb-a3c3-431e-a288-b6c336b7d3de admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:40.311117 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e52fb-a3c3-431e-a288-b6c336b7d3de admin admin] PATH_INFO: `/v3/projects/e7b6b713312249bb81e467ed2be3ad61/users/c1699bcf04214829a508a9ecc3e4481f/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:40.326776 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037e52fb-a3c3-431e-a288-b6c336b7d3de admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7b6b713312249bb81e467ed2be3ad61, user_id=c1699bcf04214829a508a9ecc3e4481f, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:40.329531 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037e52fb-a3c3-431e-a288-b6c336b7d3de admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:40.345526 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1148/2303] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:35:40 2022] PUT /identity/v3/projects/e7b6b713312249bb81e467ed2be3ad61/users/c1699bcf04214829a508a9ecc3e4481f/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:40.357657 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248482b2-4c71-431d-a07b-a056a9309a81 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:40.357946 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248482b2-4c71-431d-a07b-a056a9309a81 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:40.358209 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248482b2-4c71-431d-a07b-a056a9309a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:40.388787 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-248482b2-4c71-431d-a07b-a056a9309a81 None None] MFA Rules not processed for user `c1699bcf04214829a508a9ecc3e4481f`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:40.390044 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-248482b2-4c71-431d-a07b-a056a9309a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:40.467523 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1156/2304] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:35:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:35:40.472315 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e52fb-a3c3-431e-a288-b6c336b7d3de admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:40.543026 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e52fb-a3c3-431e-a288-b6c336b7d3de admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:40.550125 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16820f7f-d05b-4419-b012-468f0426b0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:40.550771 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16820f7f-d05b-4419-b012-468f0426b0b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:40.551082 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16820f7f-d05b-4419-b012-468f0426b0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:40.551370 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16820f7f-d05b-4419-b012-468f0426b0b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:40.552001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16820f7f-d05b-4419-b012-468f0426b0b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:40.553993 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16820f7f-d05b-4419-b012-468f0426b0b5 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:40.589489 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1149/2305] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:35:40 2022] POST /identity/v3/projects => generated 367 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:35:40.590712 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248482b2-4c71-431d-a07b-a056a9309a81 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:40.602219 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248482b2-4c71-431d-a07b-a056a9309a81 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:40.652307 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:40.663219 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7790c6a4-e896-43f7-94d5-3217842ad0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:40.663833 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790c6a4-e896-43f7-94d5-3217842ad0c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:40.664110 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790c6a4-e896-43f7-94d5-3217842ad0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:40.664386 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790c6a4-e896-43f7-94d5-3217842ad0c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:40.665025 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7790c6a4-e896-43f7-94d5-3217842ad0c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:40.666992 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7790c6a4-e896-43f7-94d5-3217842ad0c5 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:40.689974 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1157/2306] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:35:40 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:35:40.694106 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16820f7f-d05b-4419-b012-468f0426b0b5 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:40.705511 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16820f7f-d05b-4419-b012-468f0426b0b5 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:40.711076 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb41789-9ef1-4ee6-a0c1-b8f33694d50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:40.711708 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb41789-9ef1-4ee6-a0c1-b8f33694d50d admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:40.711978 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb41789-9ef1-4ee6-a0c1-b8f33694d50d admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:40.712245 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb41789-9ef1-4ee6-a0c1-b8f33694d50d admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:40.712883 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb41789-9ef1-4ee6-a0c1-b8f33694d50d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:40.714482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb41789-9ef1-4ee6-a0c1-b8f33694d50d admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:40.725139 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1150/2307] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:35:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:35:40.729490 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7790c6a4-e896-43f7-94d5-3217842ad0c5 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:40.740775 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7790c6a4-e896-43f7-94d5-3217842ad0c5 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:40.746998 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e9a158-a64f-4655-9afd-fd06c0bb4887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:40.747652 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9a158-a64f-4655-9afd-fd06c0bb4887 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:40.747925 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9a158-a64f-4655-9afd-fd06c0bb4887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:40.748192 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9a158-a64f-4655-9afd-fd06c0bb4887 admin admin] PATH_INFO: `/v3/projects/54fda39040eb4fb5b7e66b741edd0db7/users/6c4b464909a5475a8d2670826e917825/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:40.764268 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e9a158-a64f-4655-9afd-fd06c0bb4887 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54fda39040eb4fb5b7e66b741edd0db7, user_id=6c4b464909a5475a8d2670826e917825, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:40.767170 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e9a158-a64f-4655-9afd-fd06c0bb4887 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:40.781449 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1158/2308] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:35:40 2022] PUT /identity/v3/projects/54fda39040eb4fb5b7e66b741edd0db7/users/6c4b464909a5475a8d2670826e917825/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:40.785485 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb41789-9ef1-4ee6-a0c1-b8f33694d50d admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:40.791918 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:40.796083 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb41789-9ef1-4ee6-a0c1-b8f33694d50d admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:40.856897 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd33d48-26a3-4bfc-9482-32d4ae28fbfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:40.857579 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd33d48-26a3-4bfc-9482-32d4ae28fbfc admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:40.857857 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd33d48-26a3-4bfc-9482-32d4ae28fbfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:40.858126 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd33d48-26a3-4bfc-9482-32d4ae28fbfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:40.858909 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd33d48-26a3-4bfc-9482-32d4ae28fbfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:40.860523 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd33d48-26a3-4bfc-9482-32d4ae28fbfc admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:40.871256 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1151/2309] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:35:40 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:35:40.875913 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e9a158-a64f-4655-9afd-fd06c0bb4887 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:40.886417 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e9a158-a64f-4655-9afd-fd06c0bb4887 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:40.891512 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52469efd-2e6d-409a-84fa-581a4c8afc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:40.892164 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52469efd-2e6d-409a-84fa-581a4c8afc1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:40.892439 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52469efd-2e6d-409a-84fa-581a4c8afc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:40.892708 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52469efd-2e6d-409a-84fa-581a4c8afc1c admin admin] PATH_INFO: `/v3/projects/54fda39040eb4fb5b7e66b741edd0db7/users/6c4b464909a5475a8d2670826e917825/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:40.894761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52469efd-2e6d-409a-84fa-581a4c8afc1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=54fda39040eb4fb5b7e66b741edd0db7, user_id=6c4b464909a5475a8d2670826e917825, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:40.897410 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52469efd-2e6d-409a-84fa-581a4c8afc1c admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:40.907971 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1159/2310] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:35:40 2022] PUT /identity/v3/projects/54fda39040eb4fb5b7e66b741edd0db7/users/6c4b464909a5475a8d2670826e917825/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:40.918556 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaba2f2-6ef7-4e04-8260-4112e1f10fb0 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:40.918863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaba2f2-6ef7-4e04-8260-4112e1f10fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:40.919135 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaba2f2-6ef7-4e04-8260-4112e1f10fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:40.948746 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-feaba2f2-6ef7-4e04-8260-4112e1f10fb0 None None] MFA Rules not processed for user `6c4b464909a5475a8d2670826e917825`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:40.949982 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-feaba2f2-6ef7-4e04-8260-4112e1f10fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:41.021519 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1152/2311] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:35:40 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:35:41.034282 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52469efd-2e6d-409a-84fa-581a4c8afc1c admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:41.036087 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-52469efd-2e6d-409a-84fa-581a4c8afc1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:41.095323 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52469efd-2e6d-409a-84fa-581a4c8afc1c admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:41.099407 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda114e-cd37-4787-90f3-569eb22e3d7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:41.100011 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda114e-cd37-4787-90f3-569eb22e3d7b service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:41.100286 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda114e-cd37-4787-90f3-569eb22e3d7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:41.100571 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda114e-cd37-4787-90f3-569eb22e3d7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:41.110021 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda114e-cd37-4787-90f3-569eb22e3d7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:41.111509 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda114e-cd37-4787-90f3-569eb22e3d7b service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:41.123471 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1160/2312] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:35:41 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:35:41.187613 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaba2f2-6ef7-4e04-8260-4112e1f10fb0 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:41.189208 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-feaba2f2-6ef7-4e04-8260-4112e1f10fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:41.205011 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaba2f2-6ef7-4e04-8260-4112e1f10fb0 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:41.208817 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd09ec-7f3a-4f5b-b71b-6a30d7683ca0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:41.209415 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd09ec-7f3a-4f5b-b71b-6a30d7683ca0 service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:41.209683 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd09ec-7f3a-4f5b-b71b-6a30d7683ca0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:41.209945 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd09ec-7f3a-4f5b-b71b-6a30d7683ca0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:41.273345 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdd09ec-7f3a-4f5b-b71b-6a30d7683ca0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:41.274961 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdd09ec-7f3a-4f5b-b71b-6a30d7683ca0 service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:41.285516 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1153/2313] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:35:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:35:44.108259 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448afccf-ba79-4c0c-ab5b-4fbabbdd43d5 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:44.108754 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448afccf-ba79-4c0c-ab5b-4fbabbdd43d5 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:44.108919 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448afccf-ba79-4c0c-ab5b-4fbabbdd43d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:44.132390 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-448afccf-ba79-4c0c-ab5b-4fbabbdd43d5 None None] MFA Rules not processed for user `364555ec385c4dbb91907fb9c5116d2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:44.133587 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-448afccf-ba79-4c0c-ab5b-4fbabbdd43d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:44.195384 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1161/2314] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Nov 2 07:35:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:44.249768 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd09ec-7f3a-4f5b-b71b-6a30d7683ca0 service cinder] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:44.251479 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-2fdd09ec-7f3a-4f5b-b71b-6a30d7683ca0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:44.281404 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:44.327338 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd09ec-7f3a-4f5b-b71b-6a30d7683ca0 service cinder] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:44.332595 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb26e7c-aa19-4278-80c6-96de3b8b92a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '836c4044aaf74de391219289b57d2956', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:44.333248 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb26e7c-aa19-4278-80c6-96de3b8b92a3 service swift] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:44.333515 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb26e7c-aa19-4278-80c6-96de3b8b92a3 service swift] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:44.333776 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb26e7c-aa19-4278-80c6-96de3b8b92a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:44.341908 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb26e7c-aa19-4278-80c6-96de3b8b92a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:44.343523 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb26e7c-aa19-4278-80c6-96de3b8b92a3 service swift] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:44.352562 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1154/2315] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Nov 2 07:35:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:35:44.726560 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448afccf-ba79-4c0c-ab5b-4fbabbdd43d5 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:44.819303 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:44.847644 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448afccf-ba79-4c0c-ab5b-4fbabbdd43d5 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:44.853902 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af49fb-65a5-4583-b2d3-4a6d4c2ceeec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:44.854735 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af49fb-65a5-4583-b2d3-4a6d4c2ceeec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:44.855114 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af49fb-65a5-4583-b2d3-4a6d4c2ceeec admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:44.855483 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af49fb-65a5-4583-b2d3-4a6d4c2ceeec admin admin] PATH_INFO: `/v3/users/c1699bcf04214829a508a9ecc3e4481f` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:44.856927 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af49fb-65a5-4583-b2d3-4a6d4c2ceeec admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1699bcf04214829a508a9ecc3e4481f)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:44.859052 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af49fb-65a5-4583-b2d3-4a6d4c2ceeec admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:44.931973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-b0af49fb-65a5-4583-b2d3-4a6d4c2ceeec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1699bcf04214829a508a9ecc3e4481f'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:44.942039 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-b0af49fb-65a5-4583-b2d3-4a6d4c2ceeec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1699bcf04214829a508a9ecc3e4481f'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:44.943171 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-b0af49fb-65a5-4583-b2d3-4a6d4c2ceeec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1699bcf04214829a508a9ecc3e4481f'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:44.972176 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-b0af49fb-65a5-4583-b2d3-4a6d4c2ceeec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1699bcf04214829a508a9ecc3e4481f'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:44.995296 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-b0af49fb-65a5-4583-b2d3-4a6d4c2ceeec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1699bcf04214829a508a9ecc3e4481f'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:45.007065 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1162/2316] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:35:44 2022] DELETE /identity/v3/users/c1699bcf04214829a508a9ecc3e4481f => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:45.011308 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33ed1b-4bb9-4b1e-9f8f-58a8eeef87eb None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:45.011851 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33ed1b-4bb9-4b1e-9f8f-58a8eeef87eb None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:45.012346 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33ed1b-4bb9-4b1e-9f8f-58a8eeef87eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:45.038337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-ee33ed1b-4bb9-4b1e-9f8f-58a8eeef87eb None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:45.039688 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-ee33ed1b-4bb9-4b1e-9f8f-58a8eeef87eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:45.108271 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1155/2317] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:35:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:45.119331 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af49fb-65a5-4583-b2d3-4a6d4c2ceeec admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:45.120836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-b0af49fb-65a5-4583-b2d3-4a6d4c2ceeec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:45.190343 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af49fb-65a5-4583-b2d3-4a6d4c2ceeec admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:45.194264 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc3c40-2f42-4f00-b718-5947a626e001 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:45.194996 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc3c40-2f42-4f00-b718-5947a626e001 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:45.195344 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc3c40-2f42-4f00-b718-5947a626e001 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:45.195683 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc3c40-2f42-4f00-b718-5947a626e001 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:45.207877 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacc3c40-2f42-4f00-b718-5947a626e001 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:45.209343 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacc3c40-2f42-4f00-b718-5947a626e001 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:45.220978 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1163/2318] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:35:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:35:45.528067 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33ed1b-4bb9-4b1e-9f8f-58a8eeef87eb None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:45.530125 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-ee33ed1b-4bb9-4b1e-9f8f-58a8eeef87eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:45.550059 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33ed1b-4bb9-4b1e-9f8f-58a8eeef87eb None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:45.556195 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cee634-5521-49e3-8fdf-344fe3b2d7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:45.566594 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cee634-5521-49e3-8fdf-344fe3b2d7f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:45.567122 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cee634-5521-49e3-8fdf-344fe3b2d7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:45.567535 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cee634-5521-49e3-8fdf-344fe3b2d7f0 admin admin] PATH_INFO: `/v3/projects/e7b6b713312249bb81e467ed2be3ad61` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:45.577506 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cee634-5521-49e3-8fdf-344fe3b2d7f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7b6b713312249bb81e467ed2be3ad61)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:45.579942 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cee634-5521-49e3-8fdf-344fe3b2d7f0 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:45.652230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-62cee634-5521-49e3-8fdf-344fe3b2d7f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7b6b713312249bb81e467ed2be3ad61', 'request_id': 'req-62cee634-5521-49e3-8fdf-344fe3b2d7f0'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:45.659337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-62cee634-5521-49e3-8fdf-344fe3b2d7f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7b6b713312249bb81e467ed2be3ad61', 'request_id': 'req-62cee634-5521-49e3-8fdf-344fe3b2d7f0'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:45.669071 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-62cee634-5521-49e3-8fdf-344fe3b2d7f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7b6b713312249bb81e467ed2be3ad61', 'request_id': 'req-62cee634-5521-49e3-8fdf-344fe3b2d7f0'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:45.684852 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-62cee634-5521-49e3-8fdf-344fe3b2d7f0 admin admin] The token cache is being invalidate because project e7b6b713312249bb81e467ed2be3ad61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:35:45.684852 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-62cee634-5521-49e3-8fdf-344fe3b2d7f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:45.685723 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-62cee634-5521-49e3-8fdf-344fe3b2d7f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:45.690672 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1156/2319] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:35:45 2022] DELETE /identity/v3/projects/e7b6b713312249bb81e467ed2be3ad61 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:45.691734 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc3c40-2f42-4f00-b718-5947a626e001 service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:45.702472 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc3c40-2f42-4f00-b718-5947a626e001 service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:45.775415 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb7d241-e1e8-4505-9561-fcd04b5174f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:45.776302 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7d241-e1e8-4505-9561-fcd04b5174f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:45.776638 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7d241-e1e8-4505-9561-fcd04b5174f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:45.776992 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7d241-e1e8-4505-9561-fcd04b5174f8 admin admin] PATH_INFO: `/v3/users/6c4b464909a5475a8d2670826e917825` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:45.778127 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb7d241-e1e8-4505-9561-fcd04b5174f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c4b464909a5475a8d2670826e917825)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:45.780082 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb7d241-e1e8-4505-9561-fcd04b5174f8 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:45.798225 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:45.849143 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-2cb7d241-e1e8-4505-9561-fcd04b5174f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c4b464909a5475a8d2670826e917825'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:45.856899 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-2cb7d241-e1e8-4505-9561-fcd04b5174f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c4b464909a5475a8d2670826e917825'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:45.857697 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-2cb7d241-e1e8-4505-9561-fcd04b5174f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c4b464909a5475a8d2670826e917825'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:45.874438 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-2cb7d241-e1e8-4505-9561-fcd04b5174f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c4b464909a5475a8d2670826e917825'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:45.887349 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-2cb7d241-e1e8-4505-9561-fcd04b5174f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c4b464909a5475a8d2670826e917825'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:45.898910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1164/2320] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:35:45 2022] DELETE /identity/v3/users/6c4b464909a5475a8d2670826e917825 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:35:46.340194 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cee634-5521-49e3-8fdf-344fe3b2d7f0 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:46.341567 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-62cee634-5521-49e3-8fdf-344fe3b2d7f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:46.376664 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cee634-5521-49e3-8fdf-344fe3b2d7f0 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:46.419725 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:46.448564 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff2ee73-2750-455b-b9f7-19d5f6d66438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:46.449494 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2ee73-2750-455b-b9f7-19d5f6d66438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:46.449925 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2ee73-2750-455b-b9f7-19d5f6d66438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:46.450194 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2ee73-2750-455b-b9f7-19d5f6d66438 admin admin] PATH_INFO: `/v3/projects/54fda39040eb4fb5b7e66b741edd0db7` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:46.451321 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff2ee73-2750-455b-b9f7-19d5f6d66438 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54fda39040eb4fb5b7e66b741edd0db7)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:46.453384 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff2ee73-2750-455b-b9f7-19d5f6d66438 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:46.512369 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-9ff2ee73-2750-455b-b9f7-19d5f6d66438 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54fda39040eb4fb5b7e66b741edd0db7', 'request_id': 'req-9ff2ee73-2750-455b-b9f7-19d5f6d66438'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:46.522824 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-9ff2ee73-2750-455b-b9f7-19d5f6d66438 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54fda39040eb4fb5b7e66b741edd0db7', 'request_id': 'req-9ff2ee73-2750-455b-b9f7-19d5f6d66438'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:46.528883 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-9ff2ee73-2750-455b-b9f7-19d5f6d66438 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54fda39040eb4fb5b7e66b741edd0db7', 'request_id': 'req-9ff2ee73-2750-455b-b9f7-19d5f6d66438'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:46.545240 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-9ff2ee73-2750-455b-b9f7-19d5f6d66438 admin admin] The token cache is being invalidate because project 54fda39040eb4fb5b7e66b741edd0db7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:35:46.545811 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-9ff2ee73-2750-455b-b9f7-19d5f6d66438 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:46.546437 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-9ff2ee73-2750-455b-b9f7-19d5f6d66438 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:46.550738 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1157/2321] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:35:46 2022] DELETE /identity/v3/projects/54fda39040eb4fb5b7e66b741edd0db7 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:46.553471 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20881230-fb4f-4c60-89c4-85f354f319ce None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:46.554079 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20881230-fb4f-4c60-89c4-85f354f319ce None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:46.554079 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20881230-fb4f-4c60-89c4-85f354f319ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:46.576995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-20881230-fb4f-4c60-89c4-85f354f319ce None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:46.578316 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-20881230-fb4f-4c60-89c4-85f354f319ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:46.654953 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1165/2322] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:35:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:35:46.665347 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d3bf1-a005-404c-9e9c-de65af5dc78d None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:46.666267 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d3bf1-a005-404c-9e9c-de65af5dc78d None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:46.666267 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d3bf1-a005-404c-9e9c-de65af5dc78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:46.695313 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-128d3bf1-a005-404c-9e9c-de65af5dc78d None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:46.696555 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-128d3bf1-a005-404c-9e9c-de65af5dc78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:46.710688 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1158/2323] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:35:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:46.711930 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20881230-fb4f-4c60-89c4-85f354f319ce None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:46.724304 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20881230-fb4f-4c60-89c4-85f354f319ce None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:46.729181 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1460c5-924d-4521-92ed-3be9aeb1ae57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:46.730738 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1460c5-924d-4521-92ed-3be9aeb1ae57 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:46.731022 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1460c5-924d-4521-92ed-3be9aeb1ae57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:46.731290 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1460c5-924d-4521-92ed-3be9aeb1ae57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:46.731977 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1460c5-924d-4521-92ed-3be9aeb1ae57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:35:46.732401 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1460c5-924d-4521-92ed-3be9aeb1ae57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:46.733866 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1460c5-924d-4521-92ed-3be9aeb1ae57 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:46.747374 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1166/2324] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:35:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 07:35:46.754726 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d3bf1-a005-404c-9e9c-de65af5dc78d None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:46.765039 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d3bf1-a005-404c-9e9c-de65af5dc78d None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:46.771631 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d7591-89e5-4029-9682-83fa8c6a73ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:46.771631 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d7591-89e5-4029-9682-83fa8c6a73ce admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:46.771631 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d7591-89e5-4029-9682-83fa8c6a73ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:46.771631 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d7591-89e5-4029-9682-83fa8c6a73ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:46.772749 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d7591-89e5-4029-9682-83fa8c6a73ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:46.775316 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d7591-89e5-4029-9682-83fa8c6a73ce admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:46.809741 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1159/2325] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:35:46 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:35:46.814073 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1460c5-924d-4521-92ed-3be9aeb1ae57 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:46.824828 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1460c5-924d-4521-92ed-3be9aeb1ae57 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:46.897564 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf3482d-7c9f-4d91-84e9-b0dd194a68f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:46.898103 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf3482d-7c9f-4d91-84e9-b0dd194a68f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:46.898382 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf3482d-7c9f-4d91-84e9-b0dd194a68f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:46.898647 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf3482d-7c9f-4d91-84e9-b0dd194a68f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:46.899321 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf3482d-7c9f-4d91-84e9-b0dd194a68f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:46.901267 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf3482d-7c9f-4d91-84e9-b0dd194a68f0 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:46.930168 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1167/2326] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:35:46 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:35:46.931831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d7591-89e5-4029-9682-83fa8c6a73ce admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:46.946176 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d7591-89e5-4029-9682-83fa8c6a73ce admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:46.951143 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c576331a-f3a5-4b0d-8487-6ccea3e37d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:46.951789 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576331a-f3a5-4b0d-8487-6ccea3e37d37 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:46.952064 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576331a-f3a5-4b0d-8487-6ccea3e37d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:46.952336 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576331a-f3a5-4b0d-8487-6ccea3e37d37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:46.952982 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c576331a-f3a5-4b0d-8487-6ccea3e37d37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:46.954921 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c576331a-f3a5-4b0d-8487-6ccea3e37d37 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:46.969851 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1160/2327] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:35:46.971047 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf3482d-7c9f-4d91-84e9-b0dd194a68f0 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:46.982712 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf3482d-7c9f-4d91-84e9-b0dd194a68f0 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:46.987739 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae71a90-be29-48c0-bc25-b7a520a2dbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:46.988360 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae71a90-be29-48c0-bc25-b7a520a2dbb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:46.988625 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae71a90-be29-48c0-bc25-b7a520a2dbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:46.988885 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae71a90-be29-48c0-bc25-b7a520a2dbb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:46.989523 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae71a90-be29-48c0-bc25-b7a520a2dbb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:46.991148 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae71a90-be29-48c0-bc25-b7a520a2dbb8 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:46.994123 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:47.005891 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1168/2328] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 07:35:47.010950 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c576331a-f3a5-4b0d-8487-6ccea3e37d37 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:47.022816 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c576331a-f3a5-4b0d-8487-6ccea3e37d37 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:47.027184 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0c724-34f8-405d-9ec8-13fee4f0d8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:47.027887 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0c724-34f8-405d-9ec8-13fee4f0d8a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:47.028161 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0c724-34f8-405d-9ec8-13fee4f0d8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:47.028424 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0c724-34f8-405d-9ec8-13fee4f0d8a2 admin admin] PATH_INFO: `/v3/projects/d9503917187d4a3c9fc346e6b44246b6/users/042b1277484e48dbac50523b82d03f9c/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:47.049344 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0c724-34f8-405d-9ec8-13fee4f0d8a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9503917187d4a3c9fc346e6b44246b6, user_id=042b1277484e48dbac50523b82d03f9c, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:47.052199 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0c724-34f8-405d-9ec8-13fee4f0d8a2 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:47.076029 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1161/2329] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:35:47 2022] PUT /identity/v3/projects/d9503917187d4a3c9fc346e6b44246b6/users/042b1277484e48dbac50523b82d03f9c/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:47.088551 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdb8a4-c2a0-4e6f-9115-cff6b3af5078 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:47.088838 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdb8a4-c2a0-4e6f-9115-cff6b3af5078 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:47.089118 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdb8a4-c2a0-4e6f-9115-cff6b3af5078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:47.123515 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-69cdb8a4-c2a0-4e6f-9115-cff6b3af5078 None None] MFA Rules not processed for user `042b1277484e48dbac50523b82d03f9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:47.124813 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-69cdb8a4-c2a0-4e6f-9115-cff6b3af5078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:47.223898 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1169/2330] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:35:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:47.228469 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0c724-34f8-405d-9ec8-13fee4f0d8a2 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:47.298376 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0c724-34f8-405d-9ec8-13fee4f0d8a2 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:47.303401 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6e8566-590a-446f-b720-1a122f4d337e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:47.304038 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e8566-590a-446f-b720-1a122f4d337e admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:47.304312 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e8566-590a-446f-b720-1a122f4d337e admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:47.304577 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e8566-590a-446f-b720-1a122f4d337e admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:47.305216 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6e8566-590a-446f-b720-1a122f4d337e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:47.307335 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6e8566-590a-446f-b720-1a122f4d337e admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:47.317865 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:47.343014 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1162/2331] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:35:47 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:35:47.346986 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdb8a4-c2a0-4e6f-9115-cff6b3af5078 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:47.359388 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdb8a4-c2a0-4e6f-9115-cff6b3af5078 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:47.436735 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6ca3a-ed67-4a49-a9bc-e21bfda5bdf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:47.437412 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6ca3a-ed67-4a49-a9bc-e21bfda5bdf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:47.437685 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6ca3a-ed67-4a49-a9bc-e21bfda5bdf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:47.437954 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6ca3a-ed67-4a49-a9bc-e21bfda5bdf0 admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:47.438588 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb6ca3a-ed67-4a49-a9bc-e21bfda5bdf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:47.440633 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb6ca3a-ed67-4a49-a9bc-e21bfda5bdf0 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:47.475641 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1170/2332] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:35:47 2022] POST /identity/v3/users => generated 293 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:35:47.477519 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6e8566-590a-446f-b720-1a122f4d337e admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:47.494147 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6e8566-590a-446f-b720-1a122f4d337e admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:47.505144 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8a6e3-d174-4825-ba9d-d5985e062e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:47.505956 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8a6e3-d174-4825-ba9d-d5985e062e8a admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:47.506335 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8a6e3-d174-4825-ba9d-d5985e062e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:47.506744 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8a6e3-d174-4825-ba9d-d5985e062e8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:47.507790 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c8a6e3-d174-4825-ba9d-d5985e062e8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:47.510143 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c8a6e3-d174-4825-ba9d-d5985e062e8a admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:47.526706 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1163/2333] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:35:47.529762 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6ca3a-ed67-4a49-a9bc-e21bfda5bdf0 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:47.540427 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6ca3a-ed67-4a49-a9bc-e21bfda5bdf0 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:47.551108 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcef6f3-e405-4af9-b5d2-98f95be77e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:47.551741 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcef6f3-e405-4af9-b5d2-98f95be77e7b admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:47.552020 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcef6f3-e405-4af9-b5d2-98f95be77e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:47.552291 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcef6f3-e405-4af9-b5d2-98f95be77e7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:47.552966 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcef6f3-e405-4af9-b5d2-98f95be77e7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:47.554563 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcef6f3-e405-4af9-b5d2-98f95be77e7b admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:47.570644 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8a6e3-d174-4825-ba9d-d5985e062e8a admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:47.571573 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1171/2334] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:35:47.581583 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8a6e3-d174-4825-ba9d-d5985e062e8a admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:47.594524 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a0dc9-16f2-42df-b8e0-1d7e21ba2ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:47.595275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a0dc9-16f2-42df-b8e0-1d7e21ba2ca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:47.595553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a0dc9-16f2-42df-b8e0-1d7e21ba2ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:47.595825 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a0dc9-16f2-42df-b8e0-1d7e21ba2ca2 admin admin] PATH_INFO: `/v3/projects/271ebe3bdf7d47fe8fe84454abb37217/users/a1187a830f2f429e958951f5efe0f2d6/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:47.612607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45a0dc9-16f2-42df-b8e0-1d7e21ba2ca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=271ebe3bdf7d47fe8fe84454abb37217, user_id=a1187a830f2f429e958951f5efe0f2d6, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:47.615339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45a0dc9-16f2-42df-b8e0-1d7e21ba2ca2 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:47.638861 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1164/2335] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:35:47 2022] PUT /identity/v3/projects/271ebe3bdf7d47fe8fe84454abb37217/users/a1187a830f2f429e958951f5efe0f2d6/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:47.643365 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642a00c-e50d-4e79-bde1-b15797b7e359 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:47.643365 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642a00c-e50d-4e79-bde1-b15797b7e359 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:47.643365 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642a00c-e50d-4e79-bde1-b15797b7e359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:47.671830 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-f642a00c-e50d-4e79-bde1-b15797b7e359 None None] MFA Rules not processed for user `a1187a830f2f429e958951f5efe0f2d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:47.673078 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-f642a00c-e50d-4e79-bde1-b15797b7e359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:47.735297 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:47.751439 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a0dc9-16f2-42df-b8e0-1d7e21ba2ca2 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:47.752160 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1172/2336] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:35:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:47.821413 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a0dc9-16f2-42df-b8e0-1d7e21ba2ca2 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:47.826223 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d87db-0be2-42ca-91d5-2ea0050d43b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:47.826900 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d87db-0be2-42ca-91d5-2ea0050d43b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:47.827180 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d87db-0be2-42ca-91d5-2ea0050d43b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:47.827448 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d87db-0be2-42ca-91d5-2ea0050d43b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:47.830448 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d87db-0be2-42ca-91d5-2ea0050d43b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:47.830448 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d87db-0be2-42ca-91d5-2ea0050d43b0 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:47.867373 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1165/2337] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:35:47 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:35:47.869403 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f642a00c-e50d-4e79-bde1-b15797b7e359 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:47.881930 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f642a00c-e50d-4e79-bde1-b15797b7e359 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:47.945723 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c48dc7-2be4-47ac-93c3-b5716ff626b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:47.946469 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c48dc7-2be4-47ac-93c3-b5716ff626b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:47.946782 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c48dc7-2be4-47ac-93c3-b5716ff626b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:47.947076 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c48dc7-2be4-47ac-93c3-b5716ff626b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:47.947729 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c48dc7-2be4-47ac-93c3-b5716ff626b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:47.949765 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c48dc7-2be4-47ac-93c3-b5716ff626b6 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:47.979165 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1173/2338] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:35:47 2022] POST /identity/v3/users => generated 300 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:35:47.980527 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d87db-0be2-42ca-91d5-2ea0050d43b0 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:47.995537 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d87db-0be2-42ca-91d5-2ea0050d43b0 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:48.001209 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a7647-3af7-4c88-9262-7739dbdec5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:48.001824 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a7647-3af7-4c88-9262-7739dbdec5a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:48.002096 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a7647-3af7-4c88-9262-7739dbdec5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:48.002361 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a7647-3af7-4c88-9262-7739dbdec5a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:48.003040 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a7647-3af7-4c88-9262-7739dbdec5a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:48.004660 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a7647-3af7-4c88-9262-7739dbdec5a8 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:48.020998 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1166/2339] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:35:48.025450 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c48dc7-2be4-47ac-93c3-b5716ff626b6 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:48.038496 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c48dc7-2be4-47ac-93c3-b5716ff626b6 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:48.043657 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc15b824-607b-4bdf-8763-ea6de22706a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:48.044342 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15b824-607b-4bdf-8763-ea6de22706a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:48.044636 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15b824-607b-4bdf-8763-ea6de22706a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:48.044954 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15b824-607b-4bdf-8763-ea6de22706a5 admin admin] PATH_INFO: `/v3/projects/c02af693175a4f5c9c10f16153ee8db3/users/533515a7f48d4483b9538f8d0970bd0b/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:48.064189 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc15b824-607b-4bdf-8763-ea6de22706a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c02af693175a4f5c9c10f16153ee8db3, user_id=533515a7f48d4483b9538f8d0970bd0b, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:48.067104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc15b824-607b-4bdf-8763-ea6de22706a5 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:48.084142 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1174/2340] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:35:48 2022] PUT /identity/v3/projects/c02af693175a4f5c9c10f16153ee8db3/users/533515a7f48d4483b9538f8d0970bd0b/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:48.088331 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a7647-3af7-4c88-9262-7739dbdec5a8 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:48.100319 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a7647-3af7-4c88-9262-7739dbdec5a8 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:48.167015 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df6940a-f4c4-4246-a32a-5a6893081a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:48.167686 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df6940a-f4c4-4246-a32a-5a6893081a13 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:48.167964 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df6940a-f4c4-4246-a32a-5a6893081a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:48.168233 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df6940a-f4c4-4246-a32a-5a6893081a13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:48.168900 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df6940a-f4c4-4246-a32a-5a6893081a13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:48.170528 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df6940a-f4c4-4246-a32a-5a6893081a13 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:48.181820 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1167/2341] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:35:48.185959 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc15b824-607b-4bdf-8763-ea6de22706a5 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:48.197309 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc15b824-607b-4bdf-8763-ea6de22706a5 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:48.202716 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aaeb64-1d82-4086-b492-f8a14b01dde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:48.203511 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aaeb64-1d82-4086-b492-f8a14b01dde8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:48.203879 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aaeb64-1d82-4086-b492-f8a14b01dde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:48.204157 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aaeb64-1d82-4086-b492-f8a14b01dde8 admin admin] PATH_INFO: `/v3/projects/c02af693175a4f5c9c10f16153ee8db3/users/533515a7f48d4483b9538f8d0970bd0b/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:48.206094 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aaeb64-1d82-4086-b492-f8a14b01dde8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c02af693175a4f5c9c10f16153ee8db3, user_id=533515a7f48d4483b9538f8d0970bd0b, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:48.208974 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aaeb64-1d82-4086-b492-f8a14b01dde8 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:48.229655 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1175/2342] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:35:48 2022] PUT /identity/v3/projects/c02af693175a4f5c9c10f16153ee8db3/users/533515a7f48d4483b9538f8d0970bd0b/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:48.231247 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05241381-fa6f-4549-8ca4-84f8607d0201 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:48.231584 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05241381-fa6f-4549-8ca4-84f8607d0201 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:48.231854 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05241381-fa6f-4549-8ca4-84f8607d0201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:48.269911 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-05241381-fa6f-4549-8ca4-84f8607d0201 None None] MFA Rules not processed for user `533515a7f48d4483b9538f8d0970bd0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:48.271091 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-05241381-fa6f-4549-8ca4-84f8607d0201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:48.287219 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:48.357988 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1168/2343] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:35:48 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:48.370437 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aaeb64-1d82-4086-b492-f8a14b01dde8 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:48.372742 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-a6aaeb64-1d82-4086-b492-f8a14b01dde8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:48.451382 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aaeb64-1d82-4086-b492-f8a14b01dde8 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:48.455738 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542fd9b6-e4d8-48a2-9a92-adfe26950af7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:48.456366 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542fd9b6-e4d8-48a2-9a92-adfe26950af7 service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:48.456650 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542fd9b6-e4d8-48a2-9a92-adfe26950af7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:48.456919 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542fd9b6-e4d8-48a2-9a92-adfe26950af7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:48.467085 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542fd9b6-e4d8-48a2-9a92-adfe26950af7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:48.469015 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542fd9b6-e4d8-48a2-9a92-adfe26950af7 service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:48.479927 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1176/2344] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:35:48 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:35:49.150424 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05241381-fa6f-4549-8ca4-84f8607d0201 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:49.151888 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-05241381-fa6f-4549-8ca4-84f8607d0201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:49.165469 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05241381-fa6f-4549-8ca4-84f8607d0201 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:49.169257 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeedd915-1e13-4a35-a8cf-a1abf74180c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:49.169866 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeedd915-1e13-4a35-a8cf-a1abf74180c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:49.170141 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeedd915-1e13-4a35-a8cf-a1abf74180c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:49.170405 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeedd915-1e13-4a35-a8cf-a1abf74180c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:49.238669 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeedd915-1e13-4a35-a8cf-a1abf74180c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:49.240134 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeedd915-1e13-4a35-a8cf-a1abf74180c3 service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:49.253861 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1169/2345] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:35:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:35:53.465898 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542fd9b6-e4d8-48a2-9a92-adfe26950af7 service cinder] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:53.476228 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542fd9b6-e4d8-48a2-9a92-adfe26950af7 service cinder] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:53.480709 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54e4105-cfc3-4e9a-bb3a-7bb798160837 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:53.481429 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e4105-cfc3-4e9a-bb3a-7bb798160837 service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:53.481832 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e4105-cfc3-4e9a-bb3a-7bb798160837 service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:53.482196 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e4105-cfc3-4e9a-bb3a-7bb798160837 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:53.544363 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:53.557488 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54e4105-cfc3-4e9a-bb3a-7bb798160837 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:53.559225 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54e4105-cfc3-4e9a-bb3a-7bb798160837 service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:53.573020 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1177/2346] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:35:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:35:55.819030 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeedd915-1e13-4a35-a8cf-a1abf74180c3 service cinder] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:55.883154 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeedd915-1e13-4a35-a8cf-a1abf74180c3 service cinder] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:55.888010 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53beb6a5-9b27-4fdf-8a52-772453046a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:55.888643 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53beb6a5-9b27-4fdf-8a52-772453046a11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:55.888916 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53beb6a5-9b27-4fdf-8a52-772453046a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:55.889183 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53beb6a5-9b27-4fdf-8a52-772453046a11 admin admin] PATH_INFO: `/v3/users/042b1277484e48dbac50523b82d03f9c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:55.890370 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53beb6a5-9b27-4fdf-8a52-772453046a11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=042b1277484e48dbac50523b82d03f9c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:55.892476 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53beb6a5-9b27-4fdf-8a52-772453046a11 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:55.944095 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-53beb6a5-9b27-4fdf-8a52-772453046a11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '042b1277484e48dbac50523b82d03f9c'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:55.944754 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-53beb6a5-9b27-4fdf-8a52-772453046a11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '042b1277484e48dbac50523b82d03f9c'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:55.946760 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:55.952955 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-53beb6a5-9b27-4fdf-8a52-772453046a11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '042b1277484e48dbac50523b82d03f9c'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:55.966863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-53beb6a5-9b27-4fdf-8a52-772453046a11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '042b1277484e48dbac50523b82d03f9c'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:55.976253 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-53beb6a5-9b27-4fdf-8a52-772453046a11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '042b1277484e48dbac50523b82d03f9c'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:55.982814 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1170/2347] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:35:55 2022] DELETE /identity/v3/users/042b1277484e48dbac50523b82d03f9c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:55.987517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007bdbc0-dedc-41bc-9d82-ec051ee07f3b None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:55.988019 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007bdbc0-dedc-41bc-9d82-ec051ee07f3b None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:55.988132 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007bdbc0-dedc-41bc-9d82-ec051ee07f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:56.045041 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-007bdbc0-dedc-41bc-9d82-ec051ee07f3b None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:56.046268 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-007bdbc0-dedc-41bc-9d82-ec051ee07f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:56.135732 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1178/2348] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:35:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:56.146088 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53beb6a5-9b27-4fdf-8a52-772453046a11 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:56.147639 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-53beb6a5-9b27-4fdf-8a52-772453046a11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:56.215066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53beb6a5-9b27-4fdf-8a52-772453046a11 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:56.219218 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb727ce7-d4b6-4b80-8de1-88aa3d86d1b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:56.219825 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb727ce7-d4b6-4b80-8de1-88aa3d86d1b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:56.220098 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb727ce7-d4b6-4b80-8de1-88aa3d86d1b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:56.220363 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb727ce7-d4b6-4b80-8de1-88aa3d86d1b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:56.229890 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb727ce7-d4b6-4b80-8de1-88aa3d86d1b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:56.231365 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb727ce7-d4b6-4b80-8de1-88aa3d86d1b0 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:56.253736 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1171/2349] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:35:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:35:56.532756 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007bdbc0-dedc-41bc-9d82-ec051ee07f3b None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:56.534256 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-007bdbc0-dedc-41bc-9d82-ec051ee07f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:56.560623 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007bdbc0-dedc-41bc-9d82-ec051ee07f3b None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:56.565851 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fafebf4-5201-45db-b57e-1d79b9c8f7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:56.566493 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fafebf4-5201-45db-b57e-1d79b9c8f7b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:56.566914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fafebf4-5201-45db-b57e-1d79b9c8f7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:56.567182 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fafebf4-5201-45db-b57e-1d79b9c8f7b2 admin admin] PATH_INFO: `/v3/projects/d9503917187d4a3c9fc346e6b44246b6` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:56.568312 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fafebf4-5201-45db-b57e-1d79b9c8f7b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9503917187d4a3c9fc346e6b44246b6)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:56.570227 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fafebf4-5201-45db-b57e-1d79b9c8f7b2 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:56.622681 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-7fafebf4-5201-45db-b57e-1d79b9c8f7b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9503917187d4a3c9fc346e6b44246b6', 'request_id': 'req-7fafebf4-5201-45db-b57e-1d79b9c8f7b2'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:56.632694 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-7fafebf4-5201-45db-b57e-1d79b9c8f7b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9503917187d4a3c9fc346e6b44246b6', 'request_id': 'req-7fafebf4-5201-45db-b57e-1d79b9c8f7b2'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:56.639074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-7fafebf4-5201-45db-b57e-1d79b9c8f7b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9503917187d4a3c9fc346e6b44246b6', 'request_id': 'req-7fafebf4-5201-45db-b57e-1d79b9c8f7b2'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:56.652157 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-7fafebf4-5201-45db-b57e-1d79b9c8f7b2 admin admin] The token cache is being invalidate because project d9503917187d4a3c9fc346e6b44246b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:35:56.652402 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-7fafebf4-5201-45db-b57e-1d79b9c8f7b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:56.653208 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-7fafebf4-5201-45db-b57e-1d79b9c8f7b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:56.655076 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1179/2350] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:35:56 2022] DELETE /identity/v3/projects/d9503917187d4a3c9fc346e6b44246b6 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:56.659595 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb727ce7-d4b6-4b80-8de1-88aa3d86d1b0 service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:56.670688 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb727ce7-d4b6-4b80-8de1-88aa3d86d1b0 service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:56.742954 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd6e69f-753d-4f09-b40e-ca486e14f526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:56.743973 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6e69f-753d-4f09-b40e-ca486e14f526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:56.744240 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6e69f-753d-4f09-b40e-ca486e14f526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:56.744474 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6e69f-753d-4f09-b40e-ca486e14f526 admin admin] PATH_INFO: `/v3/users/a1187a830f2f429e958951f5efe0f2d6` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:56.745690 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd6e69f-753d-4f09-b40e-ca486e14f526 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1187a830f2f429e958951f5efe0f2d6)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:56.747826 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd6e69f-753d-4f09-b40e-ca486e14f526 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:56.801385 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-8fd6e69f-753d-4f09-b40e-ca486e14f526 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1187a830f2f429e958951f5efe0f2d6'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:56.802031 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-8fd6e69f-753d-4f09-b40e-ca486e14f526 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1187a830f2f429e958951f5efe0f2d6'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:56.809288 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-8fd6e69f-753d-4f09-b40e-ca486e14f526 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1187a830f2f429e958951f5efe0f2d6'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:56.811374 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:56.824299 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-8fd6e69f-753d-4f09-b40e-ca486e14f526 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1187a830f2f429e958951f5efe0f2d6'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:56.832287 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-8fd6e69f-753d-4f09-b40e-ca486e14f526 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1187a830f2f429e958951f5efe0f2d6'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:56.838219 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1172/2351] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:35:56 2022] DELETE /identity/v3/users/a1187a830f2f429e958951f5efe0f2d6 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:57.121872 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fafebf4-5201-45db-b57e-1d79b9c8f7b2 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:57.123346 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-7fafebf4-5201-45db-b57e-1d79b9c8f7b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:57.133396 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:57.147033 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fafebf4-5201-45db-b57e-1d79b9c8f7b2 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:57.230085 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041122a-4111-4658-b0f9-1d34bce614a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:57.230764 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041122a-4111-4658-b0f9-1d34bce614a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:57.231042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041122a-4111-4658-b0f9-1d34bce614a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:57.231312 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041122a-4111-4658-b0f9-1d34bce614a6 admin admin] PATH_INFO: `/v3/projects/271ebe3bdf7d47fe8fe84454abb37217` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:57.232465 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0041122a-4111-4658-b0f9-1d34bce614a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=271ebe3bdf7d47fe8fe84454abb37217)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:57.234464 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0041122a-4111-4658-b0f9-1d34bce614a6 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:57.285588 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0041122a-4111-4658-b0f9-1d34bce614a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '271ebe3bdf7d47fe8fe84454abb37217', 'request_id': 'req-0041122a-4111-4658-b0f9-1d34bce614a6'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:57.294917 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0041122a-4111-4658-b0f9-1d34bce614a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '271ebe3bdf7d47fe8fe84454abb37217', 'request_id': 'req-0041122a-4111-4658-b0f9-1d34bce614a6'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:57.300906 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0041122a-4111-4658-b0f9-1d34bce614a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '271ebe3bdf7d47fe8fe84454abb37217', 'request_id': 'req-0041122a-4111-4658-b0f9-1d34bce614a6'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:57.312112 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0041122a-4111-4658-b0f9-1d34bce614a6 admin admin] The token cache is being invalidate because project 271ebe3bdf7d47fe8fe84454abb37217 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:35:57.312420 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0041122a-4111-4658-b0f9-1d34bce614a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:57.313040 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-0041122a-4111-4658-b0f9-1d34bce614a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:57.314522 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1180/2352] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:35:57 2022] DELETE /identity/v3/projects/271ebe3bdf7d47fe8fe84454abb37217 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:57.318542 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd6e69f-753d-4f09-b40e-ca486e14f526 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:57.320061 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-8fd6e69f-753d-4f09-b40e-ca486e14f526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:57.333044 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd6e69f-753d-4f09-b40e-ca486e14f526 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:57.394950 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c7cbc-9a61-4561-a5f9-0794da53b5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:57.395580 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c7cbc-9a61-4561-a5f9-0794da53b5db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:57.395846 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c7cbc-9a61-4561-a5f9-0794da53b5db admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:57.396111 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c7cbc-9a61-4561-a5f9-0794da53b5db admin admin] PATH_INFO: `/v3/users/533515a7f48d4483b9538f8d0970bd0b` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:57.397277 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c7cbc-9a61-4561-a5f9-0794da53b5db admin admin] RBAC: Authorizing `identity:delete_user(user_id=533515a7f48d4483b9538f8d0970bd0b)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:57.399243 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c7cbc-9a61-4561-a5f9-0794da53b5db admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:57.449477 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-b31c7cbc-9a61-4561-a5f9-0794da53b5db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '533515a7f48d4483b9538f8d0970bd0b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:57.450169 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-b31c7cbc-9a61-4561-a5f9-0794da53b5db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '533515a7f48d4483b9538f8d0970bd0b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:57.458168 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-b31c7cbc-9a61-4561-a5f9-0794da53b5db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '533515a7f48d4483b9538f8d0970bd0b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:57.476235 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-b31c7cbc-9a61-4561-a5f9-0794da53b5db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '533515a7f48d4483b9538f8d0970bd0b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:57.488356 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-b31c7cbc-9a61-4561-a5f9-0794da53b5db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '533515a7f48d4483b9538f8d0970bd0b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:57.498467 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1173/2353] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:35:57 2022] DELETE /identity/v3/users/533515a7f48d4483b9538f8d0970bd0b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:57.800526 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041122a-4111-4658-b0f9-1d34bce614a6 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:57.801981 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-0041122a-4111-4658-b0f9-1d34bce614a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:57.816102 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041122a-4111-4658-b0f9-1d34bce614a6 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:57.873281 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ae9f3-e8a9-435a-ad42-282a953bdf66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:57.873922 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ae9f3-e8a9-435a-ad42-282a953bdf66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:57.874226 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ae9f3-e8a9-435a-ad42-282a953bdf66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:57.874493 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ae9f3-e8a9-435a-ad42-282a953bdf66 admin admin] PATH_INFO: `/v3/projects/c02af693175a4f5c9c10f16153ee8db3` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:57.875661 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ae9f3-e8a9-435a-ad42-282a953bdf66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c02af693175a4f5c9c10f16153ee8db3)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:57.877591 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ae9f3-e8a9-435a-ad42-282a953bdf66 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:57.916826 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:57.927109 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c66ae9f3-e8a9-435a-ad42-282a953bdf66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c02af693175a4f5c9c10f16153ee8db3', 'request_id': 'req-c66ae9f3-e8a9-435a-ad42-282a953bdf66'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:57.936117 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c66ae9f3-e8a9-435a-ad42-282a953bdf66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c02af693175a4f5c9c10f16153ee8db3', 'request_id': 'req-c66ae9f3-e8a9-435a-ad42-282a953bdf66'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:57.941892 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c66ae9f3-e8a9-435a-ad42-282a953bdf66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c02af693175a4f5c9c10f16153ee8db3', 'request_id': 'req-c66ae9f3-e8a9-435a-ad42-282a953bdf66'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:57.953657 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c66ae9f3-e8a9-435a-ad42-282a953bdf66 admin admin] The token cache is being invalidate because project c02af693175a4f5c9c10f16153ee8db3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:35:57.953940 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c66ae9f3-e8a9-435a-ad42-282a953bdf66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:57.954596 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c66ae9f3-e8a9-435a-ad42-282a953bdf66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:35:57.956149 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1181/2354] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:35:57 2022] DELETE /identity/v3/projects/c02af693175a4f5c9c10f16153ee8db3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:35:57.962588 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4759aba0-3483-4a50-b6b9-31bf19b79bcf None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:57.963149 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4759aba0-3483-4a50-b6b9-31bf19b79bcf None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:57.963149 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4759aba0-3483-4a50-b6b9-31bf19b79bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:58.013660 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-4759aba0-3483-4a50-b6b9-31bf19b79bcf None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:58.014887 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-4759aba0-3483-4a50-b6b9-31bf19b79bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:58.041450 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:58.075670 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1174/2355] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:35:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:35:58.084673 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c002cd-f4c4-444b-8f9e-b9d994b9d054 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:58.084950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c002cd-f4c4-444b-8f9e-b9d994b9d054 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:58.085219 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c002cd-f4c4-444b-8f9e-b9d994b9d054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:58.106215 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-76c002cd-f4c4-444b-8f9e-b9d994b9d054 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:58.107464 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-76c002cd-f4c4-444b-8f9e-b9d994b9d054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:58.115879 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1182/2356] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:35:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:58.120757 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4759aba0-3483-4a50-b6b9-31bf19b79bcf None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:58.130882 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4759aba0-3483-4a50-b6b9-31bf19b79bcf None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:58.135498 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e207af60-cd0a-43a0-b16f-3bd5b864e95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:58.136093 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207af60-cd0a-43a0-b16f-3bd5b864e95c admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:58.136366 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207af60-cd0a-43a0-b16f-3bd5b864e95c admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:58.136633 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207af60-cd0a-43a0-b16f-3bd5b864e95c admin admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:58.137273 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e207af60-cd0a-43a0-b16f-3bd5b864e95c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:35:58.137572 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e207af60-cd0a-43a0-b16f-3bd5b864e95c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:58.139878 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e207af60-cd0a-43a0-b16f-3bd5b864e95c admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:58.152074 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1175/2357] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:35:58.157260 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c002cd-f4c4-444b-8f9e-b9d994b9d054 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:58.171346 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c002cd-f4c4-444b-8f9e-b9d994b9d054 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:58.177149 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c1636-90bb-4423-88ca-46cb1601e8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:58.177753 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c1636-90bb-4423-88ca-46cb1601e8eb admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:58.178026 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c1636-90bb-4423-88ca-46cb1601e8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:58.178285 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c1636-90bb-4423-88ca-46cb1601e8eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:58.178936 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c1636-90bb-4423-88ca-46cb1601e8eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:58.180838 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c1636-90bb-4423-88ca-46cb1601e8eb admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:58.215515 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1183/2358] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:35:58 2022] POST /identity/v3/projects => generated 401 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:35:58.219345 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e207af60-cd0a-43a0-b16f-3bd5b864e95c admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:58.229973 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e207af60-cd0a-43a0-b16f-3bd5b864e95c admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:58.293403 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b42660-a615-4199-8e78-622a0d73d310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:58.294095 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b42660-a615-4199-8e78-622a0d73d310 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:58.294459 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b42660-a615-4199-8e78-622a0d73d310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:58.294835 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b42660-a615-4199-8e78-622a0d73d310 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:58.295810 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b42660-a615-4199-8e78-622a0d73d310 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:58.297812 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b42660-a615-4199-8e78-622a0d73d310 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:58.325089 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1176/2359] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:35:58 2022] POST /identity/v3/users => generated 301 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:35:58.326618 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c1636-90bb-4423-88ca-46cb1601e8eb admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:58.337863 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c1636-90bb-4423-88ca-46cb1601e8eb admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:58.342646 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716210cc-8e2a-4bd4-9416-446244345a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:58.343299 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716210cc-8e2a-4bd4-9416-446244345a46 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:58.343631 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716210cc-8e2a-4bd4-9416-446244345a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:58.343903 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716210cc-8e2a-4bd4-9416-446244345a46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:58.344542 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716210cc-8e2a-4bd4-9416-446244345a46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:58.346199 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716210cc-8e2a-4bd4-9416-446244345a46 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:58.357883 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1184/2360] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:35:58.362549 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b42660-a615-4199-8e78-622a0d73d310 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:58.375281 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b42660-a615-4199-8e78-622a0d73d310 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:58.384292 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e665438-73de-4dd8-8e9b-62205adf9706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:58.385128 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e665438-73de-4dd8-8e9b-62205adf9706 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:58.385399 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e665438-73de-4dd8-8e9b-62205adf9706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:58.385663 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e665438-73de-4dd8-8e9b-62205adf9706 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:58.386339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e665438-73de-4dd8-8e9b-62205adf9706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:58.388634 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e665438-73de-4dd8-8e9b-62205adf9706 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:58.402832 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1177/2361] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:35:58.407566 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716210cc-8e2a-4bd4-9416-446244345a46 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:58.418316 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716210cc-8e2a-4bd4-9416-446244345a46 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:58.423403 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9c579b-3292-4d4a-98cd-76a6c74b5ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:58.424070 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c579b-3292-4d4a-98cd-76a6c74b5ef0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:58.424347 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c579b-3292-4d4a-98cd-76a6c74b5ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:58.424618 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c579b-3292-4d4a-98cd-76a6c74b5ef0 admin admin] PATH_INFO: `/v3/projects/f4dec2bad36c49bd9e63ab6e3d762e5d/users/ce329928ddce4efaa890f6967146c462/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:58.440268 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9c579b-3292-4d4a-98cd-76a6c74b5ef0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4dec2bad36c49bd9e63ab6e3d762e5d, user_id=ce329928ddce4efaa890f6967146c462, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:58.443078 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9c579b-3292-4d4a-98cd-76a6c74b5ef0 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:58.462740 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1185/2362] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:35:58 2022] PUT /identity/v3/projects/f4dec2bad36c49bd9e63ab6e3d762e5d/users/ce329928ddce4efaa890f6967146c462/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:58.471171 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cedb1c5-28e6-4c52-b953-7009c1e03a2f None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:58.471487 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cedb1c5-28e6-4c52-b953-7009c1e03a2f None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:58.471859 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cedb1c5-28e6-4c52-b953-7009c1e03a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:58.502733 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-3cedb1c5-28e6-4c52-b953-7009c1e03a2f None None] MFA Rules not processed for user `ce329928ddce4efaa890f6967146c462`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:58.503957 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-3cedb1c5-28e6-4c52-b953-7009c1e03a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:58.573978 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1178/2363] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:35:58 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:35:58.579203 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9c579b-3292-4d4a-98cd-76a6c74b5ef0 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:58.649212 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9c579b-3292-4d4a-98cd-76a6c74b5ef0 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:58.653866 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c7770-fd59-4903-b8a7-820f05d915c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:58.654489 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c7770-fd59-4903-b8a7-820f05d915c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:58.654803 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c7770-fd59-4903-b8a7-820f05d915c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:58.655156 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c7770-fd59-4903-b8a7-820f05d915c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:58.655764 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c7770-fd59-4903-b8a7-820f05d915c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:58.657628 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c7770-fd59-4903-b8a7-820f05d915c3 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:58.689787 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1186/2364] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:35:58 2022] POST /identity/v3/projects => generated 399 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:35:58.694101 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cedb1c5-28e6-4c52-b953-7009c1e03a2f None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:58.710124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cedb1c5-28e6-4c52-b953-7009c1e03a2f None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:58.738636 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:58.778475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff05cec-c7f5-42cf-91f2-7df116ef74d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:58.779338 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff05cec-c7f5-42cf-91f2-7df116ef74d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:58.779615 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff05cec-c7f5-42cf-91f2-7df116ef74d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:58.779877 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff05cec-c7f5-42cf-91f2-7df116ef74d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:58.780564 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff05cec-c7f5-42cf-91f2-7df116ef74d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:58.783151 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff05cec-c7f5-42cf-91f2-7df116ef74d4 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:58.809710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1179/2365] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:35:58 2022] POST /identity/v3/users => generated 306 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:35:58.811037 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c7770-fd59-4903-b8a7-820f05d915c3 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:58.821585 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c7770-fd59-4903-b8a7-820f05d915c3 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:58.826243 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a8bb85-36a6-4b2f-baf3-98b60bb8480e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:58.826881 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a8bb85-36a6-4b2f-baf3-98b60bb8480e admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:58.827155 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a8bb85-36a6-4b2f-baf3-98b60bb8480e admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:58.827425 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a8bb85-36a6-4b2f-baf3-98b60bb8480e admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:58.828171 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a8bb85-36a6-4b2f-baf3-98b60bb8480e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:58.829779 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a8bb85-36a6-4b2f-baf3-98b60bb8480e admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:58.844978 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1187/2366] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:35:58.848866 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff05cec-c7f5-42cf-91f2-7df116ef74d4 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:58.858818 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff05cec-c7f5-42cf-91f2-7df116ef74d4 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:58.863812 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f714f490-1a98-49fb-b4b3-6abff9d1a091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:58.864467 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714f490-1a98-49fb-b4b3-6abff9d1a091 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:58.864737 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714f490-1a98-49fb-b4b3-6abff9d1a091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:58.865082 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714f490-1a98-49fb-b4b3-6abff9d1a091 admin admin] PATH_INFO: `/v3/projects/5781215bcc6e46f48e786639c3419e9a/users/4b959495fcb44aad882b02691b412b0d/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:58.881319 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f714f490-1a98-49fb-b4b3-6abff9d1a091 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5781215bcc6e46f48e786639c3419e9a, user_id=4b959495fcb44aad882b02691b412b0d, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:58.884015 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f714f490-1a98-49fb-b4b3-6abff9d1a091 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:58.898173 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1180/2367] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:35:58 2022] PUT /identity/v3/projects/5781215bcc6e46f48e786639c3419e9a/users/4b959495fcb44aad882b02691b412b0d/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:58.902266 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a8bb85-36a6-4b2f-baf3-98b60bb8480e admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:58.912794 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a8bb85-36a6-4b2f-baf3-98b60bb8480e admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:58.947694 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:58.975356 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e9df6-213e-4a9c-a6af-c728f329d6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:58.975968 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e9df6-213e-4a9c-a6af-c728f329d6ba admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:58.976242 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e9df6-213e-4a9c-a6af-c728f329d6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:58.976512 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e9df6-213e-4a9c-a6af-c728f329d6ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:58.977153 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e9df6-213e-4a9c-a6af-c728f329d6ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:58.978799 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e9df6-213e-4a9c-a6af-c728f329d6ba admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:58.988946 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1188/2368] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:35:58.992977 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f714f490-1a98-49fb-b4b3-6abff9d1a091 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:59.004057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f714f490-1a98-49fb-b4b3-6abff9d1a091 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:59.008670 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f027964-f413-4d24-b68c-94d54d03b244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:59.009358 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f027964-f413-4d24-b68c-94d54d03b244 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:59.009631 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f027964-f413-4d24-b68c-94d54d03b244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:59.009907 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f027964-f413-4d24-b68c-94d54d03b244 admin admin] PATH_INFO: `/v3/projects/5781215bcc6e46f48e786639c3419e9a/users/4b959495fcb44aad882b02691b412b0d/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:59.011784 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f027964-f413-4d24-b68c-94d54d03b244 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5781215bcc6e46f48e786639c3419e9a, user_id=4b959495fcb44aad882b02691b412b0d, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:59.014415 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f027964-f413-4d24-b68c-94d54d03b244 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:59.025583 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1181/2369] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:35:58 2022] PUT /identity/v3/projects/5781215bcc6e46f48e786639c3419e9a/users/4b959495fcb44aad882b02691b412b0d/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:35:59.095844 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4aaac-66e5-46f1-9c44-5e75c73dc32c None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:59.096487 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4aaac-66e5-46f1-9c44-5e75c73dc32c None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:59.096922 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4aaac-66e5-46f1-9c44-5e75c73dc32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:59.130413 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf83f0-4242-42c4-b517-8bb65f0c0b1f None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:59.130918 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf83f0-4242-42c4-b517-8bb65f0c0b1f None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:59.131254 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf83f0-4242-42c4-b517-8bb65f0c0b1f None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:59.131774 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-c3e4aaac-66e5-46f1-9c44-5e75c73dc32c None None] MFA Rules not processed for user `4b959495fcb44aad882b02691b412b0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:59.132936 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1182/2370] 10.222.0.30 () {54 vars in 921 bytes} [Wed Nov 2 07:35:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:35:59.133108 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-c3e4aaac-66e5-46f1-9c44-5e75c73dc32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:59.139310 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18b804-40f1-46ee-9d26-2ce2dcdda4f6 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:59.139844 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18b804-40f1-46ee-9d26-2ce2dcdda4f6 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:59.140187 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18b804-40f1-46ee-9d26-2ce2dcdda4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:59.193472 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-ea18b804-40f1-46ee-9d26-2ce2dcdda4f6 None None] MFA Rules not processed for user `e6495385dccc4fa48c05b1feb957846c`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:59.194794 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-ea18b804-40f1-46ee-9d26-2ce2dcdda4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:59.211962 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1189/2371] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:35:59 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:59.225967 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4aaac-66e5-46f1-9c44-5e75c73dc32c None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:59.227554 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-c3e4aaac-66e5-46f1-9c44-5e75c73dc32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:59.300594 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4aaac-66e5-46f1-9c44-5e75c73dc32c None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:59.304557 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e31979-cb74-41f3-9959-4db318bf5abf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:59.305199 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e31979-cb74-41f3-9959-4db318bf5abf service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:59.305475 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e31979-cb74-41f3-9959-4db318bf5abf service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:59.305797 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e31979-cb74-41f3-9959-4db318bf5abf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:59.307741 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1183/2372] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Nov 2 07:35:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:35:59.315694 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e31979-cb74-41f3-9959-4db318bf5abf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:59.317137 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e31979-cb74-41f3-9959-4db318bf5abf service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:59.319795 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18b804-40f1-46ee-9d26-2ce2dcdda4f6 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:59.321115 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-ea18b804-40f1-46ee-9d26-2ce2dcdda4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:59.327113 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1190/2373] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:35:59 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:35:59.342632 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:59.380886 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18b804-40f1-46ee-9d26-2ce2dcdda4f6 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:59.385178 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28980505-982f-49d6-862c-e624ac5787ec service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a051ca8154540ad93db6d028b8323a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:59.385930 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28980505-982f-49d6-862c-e624ac5787ec service glance] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:59.386245 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28980505-982f-49d6-862c-e624ac5787ec service glance] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:59.386559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28980505-982f-49d6-862c-e624ac5787ec service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:59.398457 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28980505-982f-49d6-862c-e624ac5787ec service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:59.400170 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28980505-982f-49d6-862c-e624ac5787ec service glance] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:59.411496 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e31979-cb74-41f3-9959-4db318bf5abf service cinder] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:59.412300 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1184/2374] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:35:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 07:35:59.420910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e31979-cb74-41f3-9959-4db318bf5abf service cinder] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:59.425314 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e199cb1-2d6b-46d9-bebb-583d4bf99aa1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:59.425988 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e199cb1-2d6b-46d9-bebb-583d4bf99aa1 service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:59.426324 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e199cb1-2d6b-46d9-bebb-583d4bf99aa1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:59.426750 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e199cb1-2d6b-46d9-bebb-583d4bf99aa1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:59.482368 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d98111-3394-4573-9ef1-6b24f8c57a19 None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:59.482677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d98111-3394-4573-9ef1-6b24f8c57a19 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:59.482964 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d98111-3394-4573-9ef1-6b24f8c57a19 None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:59.484421 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1185/2375] 10.222.0.30 () {54 vars in 921 bytes} [Wed Nov 2 07:35:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 07:35:59.486939 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e199cb1-2d6b-46d9-bebb-583d4bf99aa1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:59.488410 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e199cb1-2d6b-46d9-bebb-583d4bf99aa1 service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:59.488851 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6adcf3-45d3-4837-9ed7-caf42beb3538 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:59.489128 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6adcf3-45d3-4837-9ed7-caf42beb3538 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:59.489561 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6adcf3-45d3-4837-9ed7-caf42beb3538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:59.492043 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:35:59.498870 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1191/2376] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:35:59 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:35:59.511750 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-7d6adcf3-45d3-4837-9ed7-caf42beb3538 None None] MFA Rules not processed for user `65d51ec74e1e4c25bcc4365eb858bac0`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:59.513005 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-7d6adcf3-45d3-4837-9ed7-caf42beb3538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:59.567000 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1186/2377] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Nov 2 07:35:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:35:59.579174 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e199cb1-2d6b-46d9-bebb-583d4bf99aa1 service cinder] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:59.581432 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-3e199cb1-2d6b-46d9-bebb-583d4bf99aa1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:59.594095 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e199cb1-2d6b-46d9-bebb-583d4bf99aa1 service cinder] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:59.598329 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f73d5e4-49a6-4ba8-be1e-7eb185abf8df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:59.599037 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73d5e4-49a6-4ba8-be1e-7eb185abf8df service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:59.599433 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73d5e4-49a6-4ba8-be1e-7eb185abf8df service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:59.599571 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73d5e4-49a6-4ba8-be1e-7eb185abf8df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:59.607325 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f73d5e4-49a6-4ba8-be1e-7eb185abf8df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:59.608760 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f73d5e4-49a6-4ba8-be1e-7eb185abf8df service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:59.617297 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1192/2378] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:35:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:35:59.702398 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95ed6e-8181-466b-9372-d736936d307e None None] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:59.702793 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95ed6e-8181-466b-9372-d736936d307e None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:59.703092 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95ed6e-8181-466b-9372-d736936d307e None None] PATH_INFO: `/` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:59.704458 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1187/2379] 10.222.0.30 () {54 vars in 921 bytes} [Wed Nov 2 07:35:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 07:35:59.710450 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909b02c-7cc7-4ad3-bc0c-cc16876af506 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:59.710848 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909b02c-7cc7-4ad3-bc0c-cc16876af506 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:59.711192 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909b02c-7cc7-4ad3-bc0c-cc16876af506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:59.732632 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-3909b02c-7cc7-4ad3-bc0c-cc16876af506 None None] MFA Rules not processed for user `e6495385dccc4fa48c05b1feb957846c`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:35:59.734034 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-3909b02c-7cc7-4ad3-bc0c-cc16876af506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:35:59.745440 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1193/2380] 10.222.0.30 () {58 vars in 1018 bytes} [Wed Nov 2 07:35:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:35:59.756554 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f95ed6e-8181-466b-9372-d736936d307e None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:35:59.770325 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f95ed6e-8181-466b-9372-d736936d307e None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:35:59.774169 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6565b-be6c-44d5-8e95-9e91bd9adeff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:35:59.774800 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6565b-be6c-44d5-8e95-9e91bd9adeff service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:35:59.775099 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6565b-be6c-44d5-8e95-9e91bd9adeff service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:35:59.775376 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6565b-be6c-44d5-8e95-9e91bd9adeff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:35:59.792882 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d6565b-be6c-44d5-8e95-9e91bd9adeff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:35:59.794428 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d6565b-be6c-44d5-8e95-9e91bd9adeff service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:35:59.806414 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1188/2381] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:35:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:02.667510 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053da4dc-e939-4de0-97ac-828d9c1fe701 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:02.668009 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053da4dc-e939-4de0-97ac-828d9c1fe701 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:02.668009 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053da4dc-e939-4de0-97ac-828d9c1fe701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:02.688028 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-053da4dc-e939-4de0-97ac-828d9c1fe701 None None] MFA Rules not processed for user `364555ec385c4dbb91907fb9c5116d2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:02.689239 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-053da4dc-e939-4de0-97ac-828d9c1fe701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:02.748101 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1194/2382] 10.222.0.30 () {58 vars in 1009 bytes} [Wed Nov 2 07:36:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:02.763006 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6565b-be6c-44d5-8e95-9e91bd9adeff service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:02.764328 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-64d6565b-be6c-44d5-8e95-9e91bd9adeff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:02.839288 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6565b-be6c-44d5-8e95-9e91bd9adeff service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:02.844194 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0b9ec1-3ec0-48af-b2d7-d86ee2186f17 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '836c4044aaf74de391219289b57d2956', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:02.844809 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b9ec1-3ec0-48af-b2d7-d86ee2186f17 service swift] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:02.845108 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b9ec1-3ec0-48af-b2d7-d86ee2186f17 service swift] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:02.845376 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b9ec1-3ec0-48af-b2d7-d86ee2186f17 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:02.853910 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0b9ec1-3ec0-48af-b2d7-d86ee2186f17 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:02.855566 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0b9ec1-3ec0-48af-b2d7-d86ee2186f17 service swift] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:02.863761 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1189/2383] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Nov 2 07:36:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:36:02.923829 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053da4dc-e939-4de0-97ac-828d9c1fe701 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:02.936874 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053da4dc-e939-4de0-97ac-828d9c1fe701 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:02.941673 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69dd800-9e98-4aa6-bdfe-3072225201fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:02.942357 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dd800-9e98-4aa6-bdfe-3072225201fd service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:02.943356 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dd800-9e98-4aa6-bdfe-3072225201fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:02.943638 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dd800-9e98-4aa6-bdfe-3072225201fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:02.946334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-b69dd800-9e98-4aa6-bdfe-3072225201fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:03.065444 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69dd800-9e98-4aa6-bdfe-3072225201fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:03.066980 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69dd800-9e98-4aa6-bdfe-3072225201fd service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:03.078821 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1195/2384] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:03.746701 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0b9ec1-3ec0-48af-b2d7-d86ee2186f17 service swift] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:03.802435 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:03.821647 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0b9ec1-3ec0-48af-b2d7-d86ee2186f17 service swift] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:03.826928 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f0b987-310f-463e-99d9-b9a674a39ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:03.828936 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0b987-310f-463e-99d9-b9a674a39ea8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:03.828936 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0b987-310f-463e-99d9-b9a674a39ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:03.828936 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0b987-310f-463e-99d9-b9a674a39ea8 admin admin] PATH_INFO: `/v3/users/ce329928ddce4efaa890f6967146c462` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:03.829508 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f0b987-310f-463e-99d9-b9a674a39ea8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce329928ddce4efaa890f6967146c462)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:03.831793 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f0b987-310f-463e-99d9-b9a674a39ea8 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:03.909693 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-76f0b987-310f-463e-99d9-b9a674a39ea8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce329928ddce4efaa890f6967146c462'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:03.910407 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-76f0b987-310f-463e-99d9-b9a674a39ea8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce329928ddce4efaa890f6967146c462'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:03.926681 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-76f0b987-310f-463e-99d9-b9a674a39ea8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce329928ddce4efaa890f6967146c462'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:03.942461 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-76f0b987-310f-463e-99d9-b9a674a39ea8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce329928ddce4efaa890f6967146c462'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:03.958958 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-76f0b987-310f-463e-99d9-b9a674a39ea8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce329928ddce4efaa890f6967146c462'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:03.975766 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1190/2385] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:36:03 2022] DELETE /identity/v3/users/ce329928ddce4efaa890f6967146c462 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:03.980549 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e5d69-73ef-427a-b085-72081aacb272 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:03.981346 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e5d69-73ef-427a-b085-72081aacb272 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:03.981618 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e5d69-73ef-427a-b085-72081aacb272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:04.004232 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-d60e5d69-73ef-427a-b085-72081aacb272 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:04.005862 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-d60e5d69-73ef-427a-b085-72081aacb272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:04.076955 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1196/2386] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:36:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:04.088704 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f0b987-310f-463e-99d9-b9a674a39ea8 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:04.090211 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-76f0b987-310f-463e-99d9-b9a674a39ea8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:04.200728 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f0b987-310f-463e-99d9-b9a674a39ea8 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:04.211387 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9421d34-601f-4ec8-afbf-52664b5e4ea4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:04.212326 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9421d34-601f-4ec8-afbf-52664b5e4ea4 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:04.212756 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9421d34-601f-4ec8-afbf-52664b5e4ea4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:04.213175 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9421d34-601f-4ec8-afbf-52664b5e4ea4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:04.244109 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9421d34-601f-4ec8-afbf-52664b5e4ea4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:04.245824 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9421d34-601f-4ec8-afbf-52664b5e4ea4 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:04.273554 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1191/2387] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:04.873133 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e5d69-73ef-427a-b085-72081aacb272 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:04.874796 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-d60e5d69-73ef-427a-b085-72081aacb272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:04.895952 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e5d69-73ef-427a-b085-72081aacb272 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:04.901672 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c99b1-4c4f-417f-9dc5-decae4e3ed67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:04.903067 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c99b1-4c4f-417f-9dc5-decae4e3ed67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:04.903354 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c99b1-4c4f-417f-9dc5-decae4e3ed67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:04.903628 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c99b1-4c4f-417f-9dc5-decae4e3ed67 admin admin] PATH_INFO: `/v3/projects/f4dec2bad36c49bd9e63ab6e3d762e5d` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:04.904880 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c99b1-4c4f-417f-9dc5-decae4e3ed67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4dec2bad36c49bd9e63ab6e3d762e5d)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:04.906845 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c99b1-4c4f-417f-9dc5-decae4e3ed67 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:04.936351 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=100 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:04.985801 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-557c99b1-4c4f-417f-9dc5-decae4e3ed67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4dec2bad36c49bd9e63ab6e3d762e5d', 'request_id': 'req-557c99b1-4c4f-417f-9dc5-decae4e3ed67'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:05.009405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-557c99b1-4c4f-417f-9dc5-decae4e3ed67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4dec2bad36c49bd9e63ab6e3d762e5d', 'request_id': 'req-557c99b1-4c4f-417f-9dc5-decae4e3ed67'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:05.023684 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-557c99b1-4c4f-417f-9dc5-decae4e3ed67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4dec2bad36c49bd9e63ab6e3d762e5d', 'request_id': 'req-557c99b1-4c4f-417f-9dc5-decae4e3ed67'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:05.037365 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-557c99b1-4c4f-417f-9dc5-decae4e3ed67 admin admin] The token cache is being invalidate because project f4dec2bad36c49bd9e63ab6e3d762e5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:36:05.037951 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-557c99b1-4c4f-417f-9dc5-decae4e3ed67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:05.038843 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-557c99b1-4c4f-417f-9dc5-decae4e3ed67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:05.040427 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1197/2388] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:36:04 2022] DELETE /identity/v3/projects/f4dec2bad36c49bd9e63ab6e3d762e5d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:05.044679 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9421d34-601f-4ec8-afbf-52664b5e4ea4 service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:05.055759 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9421d34-601f-4ec8-afbf-52664b5e4ea4 service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:05.149405 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90651ba-04aa-4493-84ad-438d9f131445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:05.149837 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90651ba-04aa-4493-84ad-438d9f131445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:05.150114 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90651ba-04aa-4493-84ad-438d9f131445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:05.150383 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90651ba-04aa-4493-84ad-438d9f131445 admin admin] PATH_INFO: `/v3/users/4b959495fcb44aad882b02691b412b0d` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:05.151923 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90651ba-04aa-4493-84ad-438d9f131445 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b959495fcb44aad882b02691b412b0d)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:05.154059 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90651ba-04aa-4493-84ad-438d9f131445 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:05.169348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:05.245424 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-d90651ba-04aa-4493-84ad-438d9f131445 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b959495fcb44aad882b02691b412b0d'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:05.246120 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-d90651ba-04aa-4493-84ad-438d9f131445 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b959495fcb44aad882b02691b412b0d'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:05.255679 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-d90651ba-04aa-4493-84ad-438d9f131445 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b959495fcb44aad882b02691b412b0d'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:05.274097 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-d90651ba-04aa-4493-84ad-438d9f131445 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b959495fcb44aad882b02691b412b0d'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:05.284677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-d90651ba-04aa-4493-84ad-438d9f131445 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b959495fcb44aad882b02691b412b0d'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:05.292475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1192/2389] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:36:05 2022] DELETE /identity/v3/users/4b959495fcb44aad882b02691b412b0d => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:05.738936 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c99b1-4c4f-417f-9dc5-decae4e3ed67 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:05.742900 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-557c99b1-4c4f-417f-9dc5-decae4e3ed67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:05.759148 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c99b1-4c4f-417f-9dc5-decae4e3ed67 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:05.845680 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b16bef-1c39-4316-9e3b-9d09f650ad2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:05.846453 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b16bef-1c39-4316-9e3b-9d09f650ad2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:05.846830 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b16bef-1c39-4316-9e3b-9d09f650ad2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:05.847153 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b16bef-1c39-4316-9e3b-9d09f650ad2b admin admin] PATH_INFO: `/v3/projects/5781215bcc6e46f48e786639c3419e9a` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:05.848412 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b16bef-1c39-4316-9e3b-9d09f650ad2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5781215bcc6e46f48e786639c3419e9a)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:05.850792 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b16bef-1c39-4316-9e3b-9d09f650ad2b admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:05.909118 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c9b16bef-1c39-4316-9e3b-9d09f650ad2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5781215bcc6e46f48e786639c3419e9a', 'request_id': 'req-c9b16bef-1c39-4316-9e3b-9d09f650ad2b'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:05.925380 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c9b16bef-1c39-4316-9e3b-9d09f650ad2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5781215bcc6e46f48e786639c3419e9a', 'request_id': 'req-c9b16bef-1c39-4316-9e3b-9d09f650ad2b'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:05.932938 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c9b16bef-1c39-4316-9e3b-9d09f650ad2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5781215bcc6e46f48e786639c3419e9a', 'request_id': 'req-c9b16bef-1c39-4316-9e3b-9d09f650ad2b'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:05.955079 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c9b16bef-1c39-4316-9e3b-9d09f650ad2b admin admin] The token cache is being invalidate because project 5781215bcc6e46f48e786639c3419e9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:36:05.955551 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c9b16bef-1c39-4316-9e3b-9d09f650ad2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:05.956275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c9b16bef-1c39-4316-9e3b-9d09f650ad2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:05.958606 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1198/2390] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:36:05 2022] DELETE /identity/v3/projects/5781215bcc6e46f48e786639c3419e9a => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:05.963366 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d588f-1b0e-4cf1-a337-cc47c5e6072d None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:05.963647 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d588f-1b0e-4cf1-a337-cc47c5e6072d None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:05.963919 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d588f-1b0e-4cf1-a337-cc47c5e6072d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:06.002848 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-371d588f-1b0e-4cf1-a337-cc47c5e6072d None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:06.004235 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-371d588f-1b0e-4cf1-a337-cc47c5e6072d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:06.083948 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1193/2391] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:06.093129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db4bc5-34f8-475b-8226-2a8c148b3e6e None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:06.093647 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db4bc5-34f8-475b-8226-2a8c148b3e6e None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:06.093728 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db4bc5-34f8-475b-8226-2a8c148b3e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:06.116286 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-88db4bc5-34f8-475b-8226-2a8c148b3e6e None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:06.117523 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-88db4bc5-34f8-475b-8226-2a8c148b3e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:06.126396 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1199/2392] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:36:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:06.131164 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d588f-1b0e-4cf1-a337-cc47c5e6072d None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:06.141337 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d588f-1b0e-4cf1-a337-cc47c5e6072d None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:06.146306 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd6125-6d6f-46cc-84ed-eb41cbf13c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:06.147011 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd6125-6d6f-46cc-84ed-eb41cbf13c81 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:06.147366 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd6125-6d6f-46cc-84ed-eb41cbf13c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:06.147565 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd6125-6d6f-46cc-84ed-eb41cbf13c81 admin admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:06.148248 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bd6125-6d6f-46cc-84ed-eb41cbf13c81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:36:06.148550 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bd6125-6d6f-46cc-84ed-eb41cbf13c81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:06.150194 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bd6125-6d6f-46cc-84ed-eb41cbf13c81 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:06.168653 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1194/2393] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:36:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:36:06.170104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db4bc5-34f8-475b-8226-2a8c148b3e6e None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:06.180835 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db4bc5-34f8-475b-8226-2a8c148b3e6e None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:06.185879 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de1ba7-3fd2-411b-8dde-7eea6fdddbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:06.186503 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de1ba7-3fd2-411b-8dde-7eea6fdddbbd admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:06.187066 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de1ba7-3fd2-411b-8dde-7eea6fdddbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:06.187435 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de1ba7-3fd2-411b-8dde-7eea6fdddbbd admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:06.188220 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de1ba7-3fd2-411b-8dde-7eea6fdddbbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:06.190196 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de1ba7-3fd2-411b-8dde-7eea6fdddbbd admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:06.223159 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1200/2394] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:36:06 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:36:06.227546 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd6125-6d6f-46cc-84ed-eb41cbf13c81 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:06.237767 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd6125-6d6f-46cc-84ed-eb41cbf13c81 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:06.298122 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab49a6f-437d-4803-b83c-d4b8244efa96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:06.298831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab49a6f-437d-4803-b83c-d4b8244efa96 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:06.299114 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab49a6f-437d-4803-b83c-d4b8244efa96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:06.299385 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab49a6f-437d-4803-b83c-d4b8244efa96 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:06.300033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab49a6f-437d-4803-b83c-d4b8244efa96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:06.302020 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab49a6f-437d-4803-b83c-d4b8244efa96 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:06.326539 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1195/2395] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:36:06 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 07:36:06.330954 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de1ba7-3fd2-411b-8dde-7eea6fdddbbd admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:06.342588 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de1ba7-3fd2-411b-8dde-7eea6fdddbbd admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:06.347249 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14496638-7a1a-44a2-bd5f-22119e75508e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:06.347860 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14496638-7a1a-44a2-bd5f-22119e75508e admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:06.348130 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14496638-7a1a-44a2-bd5f-22119e75508e admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:06.348391 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14496638-7a1a-44a2-bd5f-22119e75508e admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:06.349024 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14496638-7a1a-44a2-bd5f-22119e75508e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:06.350623 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14496638-7a1a-44a2-bd5f-22119e75508e admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:06.352106 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:06.365274 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1201/2396] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:06.366500 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab49a6f-437d-4803-b83c-d4b8244efa96 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:06.377628 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab49a6f-437d-4803-b83c-d4b8244efa96 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:06.382232 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7dd2c6-1e8e-42a3-9f4b-70960e6c0f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:06.382843 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7dd2c6-1e8e-42a3-9f4b-70960e6c0f7b admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:06.383112 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7dd2c6-1e8e-42a3-9f4b-70960e6c0f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:06.383364 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7dd2c6-1e8e-42a3-9f4b-70960e6c0f7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:06.384421 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7dd2c6-1e8e-42a3-9f4b-70960e6c0f7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:06.386034 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7dd2c6-1e8e-42a3-9f4b-70960e6c0f7b admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:06.387629 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:06.399793 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1196/2397] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:06.404015 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14496638-7a1a-44a2-bd5f-22119e75508e admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:06.414741 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14496638-7a1a-44a2-bd5f-22119e75508e admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:06.419692 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93da5c00-cb10-42d6-be41-5ff681ce3cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:06.420397 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da5c00-cb10-42d6-be41-5ff681ce3cc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:06.420673 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da5c00-cb10-42d6-be41-5ff681ce3cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:06.420942 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da5c00-cb10-42d6-be41-5ff681ce3cc2 admin admin] PATH_INFO: `/v3/projects/cbdc6c817c8f4f2aa447cb16b6f306a6/users/024449e9be344f37a3829e1d55b00171/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:06.439054 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93da5c00-cb10-42d6-be41-5ff681ce3cc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbdc6c817c8f4f2aa447cb16b6f306a6, user_id=024449e9be344f37a3829e1d55b00171, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:06.442325 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93da5c00-cb10-42d6-be41-5ff681ce3cc2 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:06.462798 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1202/2398] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:36:06 2022] PUT /identity/v3/projects/cbdc6c817c8f4f2aa447cb16b6f306a6/users/024449e9be344f37a3829e1d55b00171/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:06.473258 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe3ad7-68f3-4163-96a7-11c464cf21ec None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:06.473560 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe3ad7-68f3-4163-96a7-11c464cf21ec None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:06.473795 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe3ad7-68f3-4163-96a7-11c464cf21ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:06.505147 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-d4fe3ad7-68f3-4163-96a7-11c464cf21ec None None] MFA Rules not processed for user `024449e9be344f37a3829e1d55b00171`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:06.506555 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-d4fe3ad7-68f3-4163-96a7-11c464cf21ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:06.583606 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1197/2399] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:36:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:06.601882 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93da5c00-cb10-42d6-be41-5ff681ce3cc2 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:06.603611 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-93da5c00-cb10-42d6-be41-5ff681ce3cc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:06.700732 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93da5c00-cb10-42d6-be41-5ff681ce3cc2 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:06.704622 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10656382-1813-418f-aaa9-f1d9a8bb6955 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:06.705367 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10656382-1813-418f-aaa9-f1d9a8bb6955 service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:06.705707 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10656382-1813-418f-aaa9-f1d9a8bb6955 service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:06.706042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10656382-1813-418f-aaa9-f1d9a8bb6955 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:06.716368 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10656382-1813-418f-aaa9-f1d9a8bb6955 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:06.718027 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10656382-1813-418f-aaa9-f1d9a8bb6955 service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:06.729276 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1203/2400] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:06.749648 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe3ad7-68f3-4163-96a7-11c464cf21ec None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:06.821850 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe3ad7-68f3-4163-96a7-11c464cf21ec None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:06.826636 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5e8a5-1b8d-41bf-bdae-07e045b41c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:06.827314 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5e8a5-1b8d-41bf-bdae-07e045b41c4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:06.827589 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5e8a5-1b8d-41bf-bdae-07e045b41c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:06.827872 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5e8a5-1b8d-41bf-bdae-07e045b41c4a admin admin] PATH_INFO: `/v3/users/024449e9be344f37a3829e1d55b00171` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:06.829121 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e5e8a5-1b8d-41bf-bdae-07e045b41c4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=024449e9be344f37a3829e1d55b00171)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:06.831154 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e5e8a5-1b8d-41bf-bdae-07e045b41c4a admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:06.890435 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-78e5e8a5-1b8d-41bf-bdae-07e045b41c4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '024449e9be344f37a3829e1d55b00171'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:06.891237 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-78e5e8a5-1b8d-41bf-bdae-07e045b41c4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '024449e9be344f37a3829e1d55b00171'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:06.899101 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-78e5e8a5-1b8d-41bf-bdae-07e045b41c4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '024449e9be344f37a3829e1d55b00171'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:06.912839 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-78e5e8a5-1b8d-41bf-bdae-07e045b41c4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '024449e9be344f37a3829e1d55b00171'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:06.921615 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-78e5e8a5-1b8d-41bf-bdae-07e045b41c4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '024449e9be344f37a3829e1d55b00171'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:06.927827 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1198/2401] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:36:06 2022] DELETE /identity/v3/users/024449e9be344f37a3829e1d55b00171 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:06.932502 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a9ff8-20b6-4e3c-86f8-2552e0182033 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:06.932781 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a9ff8-20b6-4e3c-86f8-2552e0182033 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:06.933052 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a9ff8-20b6-4e3c-86f8-2552e0182033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:06.952941 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-8d2a9ff8-20b6-4e3c-86f8-2552e0182033 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:06.954308 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-8d2a9ff8-20b6-4e3c-86f8-2552e0182033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:07.033772 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5e8a5-1b8d-41bf-bdae-07e045b41c4a admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:07.035504 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-78e5e8a5-1b8d-41bf-bdae-07e045b41c4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:07.036081 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1204/2402] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:36:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 07:36:07.043701 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:07.122207 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5e8a5-1b8d-41bf-bdae-07e045b41c4a admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:07.126083 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ed8d4-edef-4f56-8c1e-5c1c8a612ada service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:07.126911 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ed8d4-edef-4f56-8c1e-5c1c8a612ada service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:07.127208 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ed8d4-edef-4f56-8c1e-5c1c8a612ada service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:07.127455 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ed8d4-edef-4f56-8c1e-5c1c8a612ada service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:07.137739 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ed8d4-edef-4f56-8c1e-5c1c8a612ada service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:07.139709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ed8d4-edef-4f56-8c1e-5c1c8a612ada service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:07.151782 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1199/2403] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:07.765547 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a9ff8-20b6-4e3c-86f8-2552e0182033 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:07.767266 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-8d2a9ff8-20b6-4e3c-86f8-2552e0182033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:07.781580 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a9ff8-20b6-4e3c-86f8-2552e0182033 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:07.786850 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d7bdf0-2c6e-4cb6-bc7c-e606f3653e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:07.787600 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d7bdf0-2c6e-4cb6-bc7c-e606f3653e40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:07.787892 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d7bdf0-2c6e-4cb6-bc7c-e606f3653e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:07.788652 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d7bdf0-2c6e-4cb6-bc7c-e606f3653e40 admin admin] PATH_INFO: `/v3/projects/cbdc6c817c8f4f2aa447cb16b6f306a6` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:07.789638 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d7bdf0-2c6e-4cb6-bc7c-e606f3653e40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbdc6c817c8f4f2aa447cb16b6f306a6)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:07.791870 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d7bdf0-2c6e-4cb6-bc7c-e606f3653e40 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:07.872877 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-04d7bdf0-2c6e-4cb6-bc7c-e606f3653e40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbdc6c817c8f4f2aa447cb16b6f306a6', 'request_id': 'req-04d7bdf0-2c6e-4cb6-bc7c-e606f3653e40'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:07.887744 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-04d7bdf0-2c6e-4cb6-bc7c-e606f3653e40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbdc6c817c8f4f2aa447cb16b6f306a6', 'request_id': 'req-04d7bdf0-2c6e-4cb6-bc7c-e606f3653e40'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:07.896530 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-04d7bdf0-2c6e-4cb6-bc7c-e606f3653e40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbdc6c817c8f4f2aa447cb16b6f306a6', 'request_id': 'req-04d7bdf0-2c6e-4cb6-bc7c-e606f3653e40'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:07.906241 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:07.912470 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-04d7bdf0-2c6e-4cb6-bc7c-e606f3653e40 admin admin] The token cache is being invalidate because project cbdc6c817c8f4f2aa447cb16b6f306a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:36:07.912772 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-04d7bdf0-2c6e-4cb6-bc7c-e606f3653e40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:07.913643 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-04d7bdf0-2c6e-4cb6-bc7c-e606f3653e40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:07.926684 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1205/2404] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:36:07 2022] DELETE /identity/v3/projects/cbdc6c817c8f4f2aa447cb16b6f306a6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:07.932652 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612edd93-8a11-4e2e-8e17-72be660bd101 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:07.932652 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612edd93-8a11-4e2e-8e17-72be660bd101 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:07.932652 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612edd93-8a11-4e2e-8e17-72be660bd101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:07.953219 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-612edd93-8a11-4e2e-8e17-72be660bd101 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:07.954444 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-612edd93-8a11-4e2e-8e17-72be660bd101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:08.016882 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1200/2405] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:36:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:08.026049 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf7b8a-3879-47ee-8232-67912dcb01f1 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:08.026336 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf7b8a-3879-47ee-8232-67912dcb01f1 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:08.026607 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf7b8a-3879-47ee-8232-67912dcb01f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:08.047315 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-badf7b8a-3879-47ee-8232-67912dcb01f1 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:08.048845 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-badf7b8a-3879-47ee-8232-67912dcb01f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:08.057219 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1206/2406] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:08.061652 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612edd93-8a11-4e2e-8e17-72be660bd101 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:08.072075 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612edd93-8a11-4e2e-8e17-72be660bd101 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:08.076715 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ebb07-0513-4999-8acf-bfddfc115eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:08.077280 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ebb07-0513-4999-8acf-bfddfc115eff admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:08.077556 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ebb07-0513-4999-8acf-bfddfc115eff admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:08.077821 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ebb07-0513-4999-8acf-bfddfc115eff admin admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:08.078472 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05ebb07-0513-4999-8acf-bfddfc115eff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 07:36:08.078792 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05ebb07-0513-4999-8acf-bfddfc115eff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:08.082693 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05ebb07-0513-4999-8acf-bfddfc115eff admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:08.096661 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1201/2407] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 07:36:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 07:36:08.097839 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf7b8a-3879-47ee-8232-67912dcb01f1 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:08.108325 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf7b8a-3879-47ee-8232-67912dcb01f1 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:08.113338 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982cb14-ab1e-4b90-9e6e-021621aaf445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:08.114029 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982cb14-ab1e-4b90-9e6e-021621aaf445 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:08.114232 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982cb14-ab1e-4b90-9e6e-021621aaf445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:08.114582 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982cb14-ab1e-4b90-9e6e-021621aaf445 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:08.115225 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3982cb14-ab1e-4b90-9e6e-021621aaf445 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:08.117208 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3982cb14-ab1e-4b90-9e6e-021621aaf445 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:08.154308 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1207/2408] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 07:36:08 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:36:08.155496 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ebb07-0513-4999-8acf-bfddfc115eff admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:08.166999 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ebb07-0513-4999-8acf-bfddfc115eff admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:08.238447 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff8cb0-333d-454d-b474-e6f3f42f3121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:08.239171 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff8cb0-333d-454d-b474-e6f3f42f3121 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:08.239507 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff8cb0-333d-454d-b474-e6f3f42f3121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:08.239920 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff8cb0-333d-454d-b474-e6f3f42f3121 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:08.240948 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff8cb0-333d-454d-b474-e6f3f42f3121 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:08.243679 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff8cb0-333d-454d-b474-e6f3f42f3121 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:08.276078 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1202/2409] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 07:36:08 2022] POST /identity/v3/users => generated 287 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 07:36:08.280644 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982cb14-ab1e-4b90-9e6e-021621aaf445 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:08.310296 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982cb14-ab1e-4b90-9e6e-021621aaf445 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:08.325874 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9fd4f-dd2a-47d2-aae2-4bcbb6689704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:08.326687 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9fd4f-dd2a-47d2-aae2-4bcbb6689704 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:08.327066 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9fd4f-dd2a-47d2-aae2-4bcbb6689704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:08.327431 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9fd4f-dd2a-47d2-aae2-4bcbb6689704 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:08.328184 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9fd4f-dd2a-47d2-aae2-4bcbb6689704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:08.330072 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9fd4f-dd2a-47d2-aae2-4bcbb6689704 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:08.350386 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1208/2410] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:08.354331 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff8cb0-333d-454d-b474-e6f3f42f3121 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:08.372437 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff8cb0-333d-454d-b474-e6f3f42f3121 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:08.379178 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95efb4c-4a8d-44ea-92cc-e62edafcb142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:08.380713 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95efb4c-4a8d-44ea-92cc-e62edafcb142 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:08.381140 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95efb4c-4a8d-44ea-92cc-e62edafcb142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:08.381555 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95efb4c-4a8d-44ea-92cc-e62edafcb142 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:08.382415 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95efb4c-4a8d-44ea-92cc-e62edafcb142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:08.384756 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95efb4c-4a8d-44ea-92cc-e62edafcb142 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:08.398558 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1203/2411] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 07:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 07:36:08.402524 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9fd4f-dd2a-47d2-aae2-4bcbb6689704 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:08.417261 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9fd4f-dd2a-47d2-aae2-4bcbb6689704 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:08.428949 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82644225-41fb-423c-8a81-0e8f348dfecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:08.430638 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82644225-41fb-423c-8a81-0e8f348dfecd admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:08.431105 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82644225-41fb-423c-8a81-0e8f348dfecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:08.431559 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82644225-41fb-423c-8a81-0e8f348dfecd admin admin] PATH_INFO: `/v3/projects/360031114dd0447387226397072a7f2c/users/8466340a496e49658cd6dbc03360a2dd/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:08.470892 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82644225-41fb-423c-8a81-0e8f348dfecd admin admin] RBAC: Authorizing `identity:create_grant(project_id=360031114dd0447387226397072a7f2c, user_id=8466340a496e49658cd6dbc03360a2dd, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:08.474078 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82644225-41fb-423c-8a81-0e8f348dfecd admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:08.490183 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1209/2412] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 07:36:08 2022] PUT /identity/v3/projects/360031114dd0447387226397072a7f2c/users/8466340a496e49658cd6dbc03360a2dd/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:36:08.495664 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5940a5-b065-496b-bf2e-b3996126b6f0 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:08.495945 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5940a5-b065-496b-bf2e-b3996126b6f0 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:08.496207 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5940a5-b065-496b-bf2e-b3996126b6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:08.516374 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-7b5940a5-b065-496b-bf2e-b3996126b6f0 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:08.517669 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-7b5940a5-b065-496b-bf2e-b3996126b6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:08.527412 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:08.585496 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1204/2413] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:08.595766 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82644225-41fb-423c-8a81-0e8f348dfecd admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:08.597438 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-82644225-41fb-423c-8a81-0e8f348dfecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:08.665932 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82644225-41fb-423c-8a81-0e8f348dfecd admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:08.670034 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b387b-9c20-4ee7-986e-38138eab17a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:08.670804 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b387b-9c20-4ee7-986e-38138eab17a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:08.671137 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b387b-9c20-4ee7-986e-38138eab17a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:08.671408 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b387b-9c20-4ee7-986e-38138eab17a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:08.681193 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38b387b-9c20-4ee7-986e-38138eab17a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:08.682648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38b387b-9c20-4ee7-986e-38138eab17a9 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:08.694710 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1210/2414] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:09.482744 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87326815-c60a-4b35-8f5f-6f2bbc4d631a None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:09.484380 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87326815-c60a-4b35-8f5f-6f2bbc4d631a None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:09.485486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87326815-c60a-4b35-8f5f-6f2bbc4d631a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:09.555409 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-87326815-c60a-4b35-8f5f-6f2bbc4d631a None None] MFA Rules not processed for user `8466340a496e49658cd6dbc03360a2dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:09.557046 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-87326815-c60a-4b35-8f5f-6f2bbc4d631a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:09.682709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1205/2415] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:36:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:09.692828 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b387b-9c20-4ee7-986e-38138eab17a9 service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:09.695058 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-b38b387b-9c20-4ee7-986e-38138eab17a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:09.838229 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b387b-9c20-4ee7-986e-38138eab17a9 service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:09.842202 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d88ed-978b-4416-89d7-315e492971f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:09.842889 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d88ed-978b-4416-89d7-315e492971f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:09.843275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d88ed-978b-4416-89d7-315e492971f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:09.843574 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d88ed-978b-4416-89d7-315e492971f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:09.858327 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d88ed-978b-4416-89d7-315e492971f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:09.859847 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d88ed-978b-4416-89d7-315e492971f4 service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:09.871959 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1211/2416] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:11.357554 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d98e5b-7738-4cd2-8c5e-83a8b734f5ad None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:11.358161 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d98e5b-7738-4cd2-8c5e-83a8b734f5ad None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:11.358161 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d98e5b-7738-4cd2-8c5e-83a8b734f5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:11.379416 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-21d98e5b-7738-4cd2-8c5e-83a8b734f5ad None None] MFA Rules not processed for user `8466340a496e49658cd6dbc03360a2dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:36:11.380633 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-21d98e5b-7738-4cd2-8c5e-83a8b734f5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:11.388672 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1206/2417] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:36:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:36:13.353874 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d88ed-978b-4416-89d7-315e492971f4 service cinder] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:13.355627 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-cc8d88ed-978b-4416-89d7-315e492971f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:13.411971 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:13.467775 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d88ed-978b-4416-89d7-315e492971f4 service cinder] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:13.475821 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd2e93-3ef3-49f8-bbe6-52277a894770 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932d3076fb4b426d8c83d02eb60d8063', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:13.476711 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd2e93-3ef3-49f8-bbe6-52277a894770 service placement] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:13.477093 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd2e93-3ef3-49f8-bbe6-52277a894770 service placement] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:13.477554 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd2e93-3ef3-49f8-bbe6-52277a894770 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:13.479554 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-ffcd2e93-3ef3-49f8-bbe6-52277a894770 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:13.493098 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd2e93-3ef3-49f8-bbe6-52277a894770 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:13.494752 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd2e93-3ef3-49f8-bbe6-52277a894770 service placement] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:13.506279 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1212/2418] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:16.882806 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d98e5b-7738-4cd2-8c5e-83a8b734f5ad None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:16.884260 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-21d98e5b-7738-4cd2-8c5e-83a8b734f5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:16.928738 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d98e5b-7738-4cd2-8c5e-83a8b734f5ad None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:16.932910 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48434a0f-75a9-4bee-bddd-0632a36a91e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:16.933553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48434a0f-75a9-4bee-bddd-0632a36a91e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:16.933842 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48434a0f-75a9-4bee-bddd-0632a36a91e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:16.934086 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48434a0f-75a9-4bee-bddd-0632a36a91e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:16.935928 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-48434a0f-75a9-4bee-bddd-0632a36a91e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:16.953371 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48434a0f-75a9-4bee-bddd-0632a36a91e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:16.954838 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48434a0f-75a9-4bee-bddd-0632a36a91e5 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:16.969611 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1207/2419] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:17.784634 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd2e93-3ef3-49f8-bbe6-52277a894770 service placement] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:17.788692 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-ffcd2e93-3ef3-49f8-bbe6-52277a894770 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:17.807886 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd2e93-3ef3-49f8-bbe6-52277a894770 service placement] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:17.812965 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149b82d-b5cc-4d37-be72-60a3ff806a7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:17.815166 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149b82d-b5cc-4d37-be72-60a3ff806a7c service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:17.815631 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149b82d-b5cc-4d37-be72-60a3ff806a7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:17.816130 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149b82d-b5cc-4d37-be72-60a3ff806a7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:17.819052 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-0149b82d-b5cc-4d37-be72-60a3ff806a7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:17.900639 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149b82d-b5cc-4d37-be72-60a3ff806a7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:17.902140 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149b82d-b5cc-4d37-be72-60a3ff806a7c service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:17.912858 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1213/2420] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:36:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:36:24.565959 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48434a0f-75a9-4bee-bddd-0632a36a91e5 service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:24.567476 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-48434a0f-75a9-4bee-bddd-0632a36a91e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:24.579371 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48434a0f-75a9-4bee-bddd-0632a36a91e5 service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:24.584185 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b84655-25cd-472e-b5e2-df558eb5b26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:24.584811 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b84655-25cd-472e-b5e2-df558eb5b26b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:24.585084 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b84655-25cd-472e-b5e2-df558eb5b26b admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:24.585351 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b84655-25cd-472e-b5e2-df558eb5b26b admin admin] PATH_INFO: `/v3/users/3f9fe0442c2b41fa822780b1b7b3b807` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:24.586454 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b84655-25cd-472e-b5e2-df558eb5b26b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f9fe0442c2b41fa822780b1b7b3b807)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:24.588401 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b84655-25cd-472e-b5e2-df558eb5b26b admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:24.636351 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-b0b84655-25cd-472e-b5e2-df558eb5b26b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f9fe0442c2b41fa822780b1b7b3b807'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:24.636994 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-b0b84655-25cd-472e-b5e2-df558eb5b26b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f9fe0442c2b41fa822780b1b7b3b807'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:24.644507 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-b0b84655-25cd-472e-b5e2-df558eb5b26b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f9fe0442c2b41fa822780b1b7b3b807'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:24.646369 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:24.658297 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-b0b84655-25cd-472e-b5e2-df558eb5b26b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f9fe0442c2b41fa822780b1b7b3b807'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:24.666899 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-b0b84655-25cd-472e-b5e2-df558eb5b26b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f9fe0442c2b41fa822780b1b7b3b807'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:24.673062 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1208/2421] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:36:24 2022] DELETE /identity/v3/users/3f9fe0442c2b41fa822780b1b7b3b807 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:36:24.903120 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149b82d-b5cc-4d37-be72-60a3ff806a7c service cinder] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:24.915759 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149b82d-b5cc-4d37-be72-60a3ff806a7c service cinder] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:24.974767 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2acca1-6115-475f-9732-8b7c96a2cde2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:24.975417 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2acca1-6115-475f-9732-8b7c96a2cde2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:24.975691 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2acca1-6115-475f-9732-8b7c96a2cde2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:24.975959 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2acca1-6115-475f-9732-8b7c96a2cde2 admin admin] PATH_INFO: `/v3/projects/1eab4b25d02147ef8232c5d37290005c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:24.977126 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2acca1-6115-475f-9732-8b7c96a2cde2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1eab4b25d02147ef8232c5d37290005c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:24.979078 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2acca1-6115-475f-9732-8b7c96a2cde2 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:25.043455 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-ea2acca1-6115-475f-9732-8b7c96a2cde2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1eab4b25d02147ef8232c5d37290005c', 'request_id': 'req-ea2acca1-6115-475f-9732-8b7c96a2cde2'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:25.045369 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:25.053688 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-ea2acca1-6115-475f-9732-8b7c96a2cde2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1eab4b25d02147ef8232c5d37290005c', 'request_id': 'req-ea2acca1-6115-475f-9732-8b7c96a2cde2'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:25.060912 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-ea2acca1-6115-475f-9732-8b7c96a2cde2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1eab4b25d02147ef8232c5d37290005c', 'request_id': 'req-ea2acca1-6115-475f-9732-8b7c96a2cde2'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:25.073269 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-ea2acca1-6115-475f-9732-8b7c96a2cde2 admin admin] The token cache is being invalidate because project 1eab4b25d02147ef8232c5d37290005c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:36:25.073557 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-ea2acca1-6115-475f-9732-8b7c96a2cde2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:25.074284 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-ea2acca1-6115-475f-9732-8b7c96a2cde2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:25.076241 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1214/2422] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:36:24 2022] DELETE /identity/v3/projects/1eab4b25d02147ef8232c5d37290005c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:25.080392 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b84655-25cd-472e-b5e2-df558eb5b26b admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:25.081945 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-b0b84655-25cd-472e-b5e2-df558eb5b26b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:25.095591 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b84655-25cd-472e-b5e2-df558eb5b26b admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:25.157783 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a30ecb-2bcf-4d04-a022-bb547d105b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:25.158584 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a30ecb-2bcf-4d04-a022-bb547d105b71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:25.158952 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a30ecb-2bcf-4d04-a022-bb547d105b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:25.159291 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a30ecb-2bcf-4d04-a022-bb547d105b71 admin admin] PATH_INFO: `/v3/users/a1449814a4754250852547e5dee5d09e` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:25.160597 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a30ecb-2bcf-4d04-a022-bb547d105b71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1449814a4754250852547e5dee5d09e)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:25.162620 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a30ecb-2bcf-4d04-a022-bb547d105b71 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:25.211435 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-d9a30ecb-2bcf-4d04-a022-bb547d105b71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1449814a4754250852547e5dee5d09e'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:25.212075 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-d9a30ecb-2bcf-4d04-a022-bb547d105b71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1449814a4754250852547e5dee5d09e'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:25.218903 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-d9a30ecb-2bcf-4d04-a022-bb547d105b71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1449814a4754250852547e5dee5d09e'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:25.230903 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-d9a30ecb-2bcf-4d04-a022-bb547d105b71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1449814a4754250852547e5dee5d09e'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:25.240342 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-d9a30ecb-2bcf-4d04-a022-bb547d105b71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1449814a4754250852547e5dee5d09e'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:25.246922 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1209/2423] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:36:25 2022] DELETE /identity/v3/users/a1449814a4754250852547e5dee5d09e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:25.446949 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2acca1-6115-475f-9732-8b7c96a2cde2 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:36:25.449062 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-ea2acca1-6115-475f-9732-8b7c96a2cde2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:36:25.477584 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2acca1-6115-475f-9732-8b7c96a2cde2 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:36:25.574697 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735d1dbe-e608-409c-a9f1-5b639893165e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:36:25.575749 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d1dbe-e608-409c-a9f1-5b639893165e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:36:25.576408 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d1dbe-e608-409c-a9f1-5b639893165e admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:36:25.576862 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d1dbe-e608-409c-a9f1-5b639893165e admin admin] PATH_INFO: `/v3/projects/2bdc6a5a22d149179b283e055bc2f576` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:36:25.578725 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735d1dbe-e608-409c-a9f1-5b639893165e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bdc6a5a22d149179b283e055bc2f576)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:36:25.582129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735d1dbe-e608-409c-a9f1-5b639893165e admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:36:25.665413 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-735d1dbe-e608-409c-a9f1-5b639893165e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bdc6a5a22d149179b283e055bc2f576', 'request_id': 'req-735d1dbe-e608-409c-a9f1-5b639893165e'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:25.675607 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-735d1dbe-e608-409c-a9f1-5b639893165e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bdc6a5a22d149179b283e055bc2f576', 'request_id': 'req-735d1dbe-e608-409c-a9f1-5b639893165e'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:25.683849 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-735d1dbe-e608-409c-a9f1-5b639893165e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bdc6a5a22d149179b283e055bc2f576', 'request_id': 'req-735d1dbe-e608-409c-a9f1-5b639893165e'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:25.702815 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-735d1dbe-e608-409c-a9f1-5b639893165e admin admin] The token cache is being invalidate because project 2bdc6a5a22d149179b283e055bc2f576 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:36:25.703413 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-735d1dbe-e608-409c-a9f1-5b639893165e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:25.704538 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-735d1dbe-e608-409c-a9f1-5b639893165e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:36:25.707135 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1215/2424] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:36:25 2022] DELETE /identity/v3/projects/2bdc6a5a22d149179b283e055bc2f576 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:36:35.243409 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=60,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:36:35.697396 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=57 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:12.871651 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302825da-381d-4bf9-84eb-082d69276a15 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:12.872265 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302825da-381d-4bf9-84eb-082d69276a15 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:12.872265 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302825da-381d-4bf9-84eb-082d69276a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:12.898172 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-302825da-381d-4bf9-84eb-082d69276a15 None None] MFA Rules not processed for user `364555ec385c4dbb91907fb9c5116d2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:37:12.899664 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-302825da-381d-4bf9-84eb-082d69276a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:12.964498 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1210/2425] 10.222.0.30 () {58 vars in 967 bytes} [Wed Nov 2 07:37:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:37:13.015557 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735d1dbe-e608-409c-a9f1-5b639893165e admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:37:13.017157 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-735d1dbe-e608-409c-a9f1-5b639893165e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:37:13.100408 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735d1dbe-e608-409c-a9f1-5b639893165e admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:37:13.108852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df19bb-6059-4e58-abf1-a142a80e34ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '836c4044aaf74de391219289b57d2956', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:37:13.109330 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df19bb-6059-4e58-abf1-a142a80e34ab service swift] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:37:13.109688 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df19bb-6059-4e58-abf1-a142a80e34ab service swift] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:37:13.109897 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df19bb-6059-4e58-abf1-a142a80e34ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:37:13.122122 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54df19bb-6059-4e58-abf1-a142a80e34ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:37:13.124498 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54df19bb-6059-4e58-abf1-a142a80e34ab service swift] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:37:13.138197 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1216/2426] 10.222.0.30 () {60 vars in 1384 bytes} [Wed Nov 2 07:37:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 07:37:22.956200 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=25 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:37:23.132797 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=32 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:09.154938 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302825da-381d-4bf9-84eb-082d69276a15 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:09.157123 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-302825da-381d-4bf9-84eb-082d69276a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:09.221948 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302825da-381d-4bf9-84eb-082d69276a15 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:09.225511 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee8f37-a6d1-42f1-a4e4-58ad6b83c2c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:09.226324 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee8f37-a6d1-42f1-a4e4-58ad6b83c2c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:09.226744 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee8f37-a6d1-42f1-a4e4-58ad6b83c2c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:09.227157 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee8f37-a6d1-42f1-a4e4-58ad6b83c2c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:09.229119 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-bfee8f37-a6d1-42f1-a4e4-58ad6b83c2c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:09.306860 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfee8f37-a6d1-42f1-a4e4-58ad6b83c2c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:09.308335 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfee8f37-a6d1-42f1-a4e4-58ad6b83c2c2 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:09.319701 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1211/2427] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:38:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:38:12.961131 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfce41-e648-425d-8742-dfc81a2cb959 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:12.961826 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfce41-e648-425d-8742-dfc81a2cb959 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:12.962110 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfce41-e648-425d-8742-dfc81a2cb959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:12.983454 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-a5cfce41-e648-425d-8742-dfc81a2cb959 None None] MFA Rules not processed for user `8466340a496e49658cd6dbc03360a2dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:38:12.984692 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-a5cfce41-e648-425d-8742-dfc81a2cb959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:13.054777 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1217/2428] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:38:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:38:13.690321 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee8f37-a6d1-42f1-a4e4-58ad6b83c2c2 service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:38:13.692023 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-bfee8f37-a6d1-42f1-a4e4-58ad6b83c2c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:13.703611 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee8f37-a6d1-42f1-a4e4-58ad6b83c2c2 service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:38:13.707170 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df425bc6-537b-44be-b4ce-934bd25a475f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:38:13.708149 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df425bc6-537b-44be-b4ce-934bd25a475f service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:38:13.708430 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df425bc6-537b-44be-b4ce-934bd25a475f service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:38:13.708735 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df425bc6-537b-44be-b4ce-934bd25a475f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:38:13.710489 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-df425bc6-537b-44be-b4ce-934bd25a475f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:38:13.724826 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df425bc6-537b-44be-b4ce-934bd25a475f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:38:13.726271 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df425bc6-537b-44be-b4ce-934bd25a475f service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:38:13.737576 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1212/2429] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:38:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:38:23.045460 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=25 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:38:23.732733 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=60 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:39:09.143639 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cfce41-e648-425d-8742-dfc81a2cb959 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:09.155673 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cfce41-e648-425d-8742-dfc81a2cb959 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:09.160103 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc52086e-5be2-4f08-8425-a4173e0ca302 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:09.160714 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52086e-5be2-4f08-8425-a4173e0ca302 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:09.160984 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52086e-5be2-4f08-8425-a4173e0ca302 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:09.161250 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52086e-5be2-4f08-8425-a4173e0ca302 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:09.163180 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-dc52086e-5be2-4f08-8425-a4173e0ca302 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:09.173646 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc52086e-5be2-4f08-8425-a4173e0ca302 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:09.175102 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc52086e-5be2-4f08-8425-a4173e0ca302 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:09.183724 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1218/2430] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:39:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:39:15.021713 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df425bc6-537b-44be-b4ce-934bd25a475f service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:15.023304 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-df425bc6-537b-44be-b4ce-934bd25a475f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:15.086686 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc52086e-5be2-4f08-8425-a4173e0ca302 service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:15.088928 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-dc52086e-5be2-4f08-8425-a4173e0ca302 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:15.108332 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df425bc6-537b-44be-b4ce-934bd25a475f service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:15.113723 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ca981-91ec-439b-83e1-1096f4a030ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932d3076fb4b426d8c83d02eb60d8063', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:15.114428 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ca981-91ec-439b-83e1-1096f4a030ba service placement] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:15.114746 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ca981-91ec-439b-83e1-1096f4a030ba service placement] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:15.115016 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ca981-91ec-439b-83e1-1096f4a030ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:15.117124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-500ca981-91ec-439b-83e1-1096f4a030ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:15.129537 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ca981-91ec-439b-83e1-1096f4a030ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:15.131176 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ca981-91ec-439b-83e1-1096f4a030ba service placement] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:15.142252 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1213/2431] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:39:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:39:15.164557 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc52086e-5be2-4f08-8425-a4173e0ca302 service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:15.169531 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c9c5c-9411-45dc-8626-b84c8a83ebf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:15.170438 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c9c5c-9411-45dc-8626-b84c8a83ebf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:15.170770 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c9c5c-9411-45dc-8626-b84c8a83ebf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:15.171041 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c9c5c-9411-45dc-8626-b84c8a83ebf3 admin admin] PATH_INFO: `/v3/users/8466340a496e49658cd6dbc03360a2dd` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:15.172316 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c9c5c-9411-45dc-8626-b84c8a83ebf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8466340a496e49658cd6dbc03360a2dd)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:15.174338 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c9c5c-9411-45dc-8626-b84c8a83ebf3 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:15.229332 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c96c9c5c-9411-45dc-8626-b84c8a83ebf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8466340a496e49658cd6dbc03360a2dd'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:15.236862 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c96c9c5c-9411-45dc-8626-b84c8a83ebf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8466340a496e49658cd6dbc03360a2dd'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:15.237579 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c96c9c5c-9411-45dc-8626-b84c8a83ebf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8466340a496e49658cd6dbc03360a2dd'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:15.250971 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c96c9c5c-9411-45dc-8626-b84c8a83ebf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8466340a496e49658cd6dbc03360a2dd'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:15.259931 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-c96c9c5c-9411-45dc-8626-b84c8a83ebf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8466340a496e49658cd6dbc03360a2dd'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:15.266952 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1219/2432] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 07:39:15 2022] DELETE /identity/v3/users/8466340a496e49658cd6dbc03360a2dd => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 07:39:15.446352 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ca981-91ec-439b-83e1-1096f4a030ba service placement] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:15.457506 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ca981-91ec-439b-83e1-1096f4a030ba service placement] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:15.521007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82ecda-247c-413e-ba4f-4ea6ef634e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:15.521644 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82ecda-247c-413e-ba4f-4ea6ef634e11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:15.521915 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82ecda-247c-413e-ba4f-4ea6ef634e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:15.522186 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82ecda-247c-413e-ba4f-4ea6ef634e11 admin admin] PATH_INFO: `/v3/projects/360031114dd0447387226397072a7f2c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:15.523297 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f82ecda-247c-413e-ba4f-4ea6ef634e11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=360031114dd0447387226397072a7f2c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:15.525195 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f82ecda-247c-413e-ba4f-4ea6ef634e11 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:15.572711 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-8f82ecda-247c-413e-ba4f-4ea6ef634e11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '360031114dd0447387226397072a7f2c', 'request_id': 'req-8f82ecda-247c-413e-ba4f-4ea6ef634e11'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:15.579325 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-8f82ecda-247c-413e-ba4f-4ea6ef634e11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '360031114dd0447387226397072a7f2c', 'request_id': 'req-8f82ecda-247c-413e-ba4f-4ea6ef634e11'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:15.586345 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-8f82ecda-247c-413e-ba4f-4ea6ef634e11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '360031114dd0447387226397072a7f2c', 'request_id': 'req-8f82ecda-247c-413e-ba4f-4ea6ef634e11'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:15.602740 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-8f82ecda-247c-413e-ba4f-4ea6ef634e11 admin admin] The token cache is being invalidate because project 360031114dd0447387226397072a7f2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 07:39:15.603129 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-8f82ecda-247c-413e-ba4f-4ea6ef634e11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:15.603968 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-8f82ecda-247c-413e-ba4f-4ea6ef634e11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 07:39:15.605814 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1214/2433] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 07:39:15 2022] DELETE /identity/v3/projects/360031114dd0447387226397072a7f2c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 07:39:21.984410 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c9c5c-9411-45dc-8626-b84c8a83ebf3 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:39:21.986095 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-c96c9c5c-9411-45dc-8626-b84c8a83ebf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:22.060884 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:39:22.066279 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c9c5c-9411-45dc-8626-b84c8a83ebf3 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:39:22.135382 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0fc4f-3f1d-4c77-aaf9-c8dba453ac3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:39:22.135996 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0fc4f-3f1d-4c77-aaf9-c8dba453ac3e service nova] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:39:22.136273 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0fc4f-3f1d-4c77-aaf9-c8dba453ac3e service nova] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:39:22.136530 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0fc4f-3f1d-4c77-aaf9-c8dba453ac3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:39:22.138496 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-e4b0fc4f-3f1d-4c77-aaf9-c8dba453ac3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:39:22.200193 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b0fc4f-3f1d-4c77-aaf9-c8dba453ac3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:39:22.202021 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b0fc4f-3f1d-4c77-aaf9-c8dba453ac3e service nova] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:39:22.211577 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1220/2434] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:39:21 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:39:25.600810 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:39:32.207059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=54 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:41:17.098961 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82ecda-247c-413e-ba4f-4ea6ef634e11 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:41:17.100542 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-8f82ecda-247c-413e-ba4f-4ea6ef634e11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:17.193965 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82ecda-247c-413e-ba4f-4ea6ef634e11 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:41:17.198038 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38a99d-83a1-448e-bbf9-5b3a3e899492 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:41:17.198671 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38a99d-83a1-448e-bbf9-5b3a3e899492 service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:41:17.198965 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38a99d-83a1-448e-bbf9-5b3a3e899492 service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:41:17.199233 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38a99d-83a1-448e-bbf9-5b3a3e899492 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:41:17.200988 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-4c38a99d-83a1-448e-bbf9-5b3a3e899492 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:41:17.295470 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38a99d-83a1-448e-bbf9-5b3a3e899492 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:41:17.296950 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38a99d-83a1-448e-bbf9-5b3a3e899492 service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:41:17.322079 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1215/2435] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:41:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:41:27.317357 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=70 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:42:10.481467 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0fc4f-3f1d-4c77-aaf9-c8dba453ac3e service nova] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:42:10.483075 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-e4b0fc4f-3f1d-4c77-aaf9-c8dba453ac3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:42:10.554620 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0fc4f-3f1d-4c77-aaf9-c8dba453ac3e service nova] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:42:10.560194 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b408a8dd-4d21-414b-9324-f914c2c65dd6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932d3076fb4b426d8c83d02eb60d8063', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:42:10.560797 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408a8dd-4d21-414b-9324-f914c2c65dd6 service placement] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:42:10.561071 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408a8dd-4d21-414b-9324-f914c2c65dd6 service placement] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:42:10.561342 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408a8dd-4d21-414b-9324-f914c2c65dd6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:42:10.563168 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-b408a8dd-4d21-414b-9324-f914c2c65dd6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:42:10.574925 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b408a8dd-4d21-414b-9324-f914c2c65dd6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:42:10.576508 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b408a8dd-4d21-414b-9324-f914c2c65dd6 service placement] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:42:10.587601 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1221/2436] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:42:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:42:20.582793 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=32 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:44:16.170757 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38a99d-83a1-448e-bbf9-5b3a3e899492 service cinder] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:44:16.172387 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-4c38a99d-83a1-448e-bbf9-5b3a3e899492 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:44:16.233510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38a99d-83a1-448e-bbf9-5b3a3e899492 service cinder] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:44:16.237743 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5848cb12-2afd-45bd-b3bb-429c7247f0c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:44:16.238339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848cb12-2afd-45bd-b3bb-429c7247f0c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:44:16.238613 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848cb12-2afd-45bd-b3bb-429c7247f0c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:44:16.238903 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848cb12-2afd-45bd-b3bb-429c7247f0c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:44:16.241275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-5848cb12-2afd-45bd-b3bb-429c7247f0c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:44:16.254160 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5848cb12-2afd-45bd-b3bb-429c7247f0c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:44:16.256702 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5848cb12-2afd-45bd-b3bb-429c7247f0c8 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:44:16.265589 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1216/2437] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:44:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:44:22.634807 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b408a8dd-4d21-414b-9324-f914c2c65dd6 service placement] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:44:22.636310 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-b408a8dd-4d21-414b-9324-f914c2c65dd6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:44:22.652519 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b408a8dd-4d21-414b-9324-f914c2c65dd6 service placement] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:44:22.656523 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2226d4d-9814-4a1b-b516-3f4c272ff845 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:44:22.657124 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2226d4d-9814-4a1b-b516-3f4c272ff845 service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:44:22.657391 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2226d4d-9814-4a1b-b516-3f4c272ff845 service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:44:22.657655 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2226d4d-9814-4a1b-b516-3f4c272ff845 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:44:22.667308 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2226d4d-9814-4a1b-b516-3f4c272ff845 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:44:22.668759 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2226d4d-9814-4a1b-b516-3f4c272ff845 service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:44:22.697342 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1222/2438] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:44:22 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:44:26.267188 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=26 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:44:32.694693 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=13 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:46:17.514020 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5848cb12-2afd-45bd-b3bb-429c7247f0c8 service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:46:17.538401 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5848cb12-2afd-45bd-b3bb-429c7247f0c8 service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:46:17.542502 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0fd51-1b40-4c86-bc87-0b0d155cd00e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:46:17.543128 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0fd51-1b40-4c86-bc87-0b0d155cd00e service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:46:17.543401 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0fd51-1b40-4c86-bc87-0b0d155cd00e service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:46:17.543670 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0fd51-1b40-4c86-bc87-0b0d155cd00e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:46:17.567987 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f0fd51-1b40-4c86-bc87-0b0d155cd00e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:46:17.569489 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f0fd51-1b40-4c86-bc87-0b0d155cd00e service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:46:17.579610 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1217/2439] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:46:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:46:27.575077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=17 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:47:16.468367 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2226d4d-9814-4a1b-b516-3f4c272ff845 service cinder] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:47:16.536391 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2226d4d-9814-4a1b-b516-3f4c272ff845 service cinder] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:47:16.542330 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb79ef9-705f-454a-8998-f6310f494895 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932d3076fb4b426d8c83d02eb60d8063', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:47:16.542957 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb79ef9-705f-454a-8998-f6310f494895 service placement] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:47:16.543228 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb79ef9-705f-454a-8998-f6310f494895 service placement] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:47:16.543497 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb79ef9-705f-454a-8998-f6310f494895 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:47:16.554940 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb79ef9-705f-454a-8998-f6310f494895 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:47:16.557598 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb79ef9-705f-454a-8998-f6310f494895 service placement] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:47:16.574287 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1223/2440] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:47:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:47:26.567016 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=27 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:48:19.879503 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0fd51-1b40-4c86-bc87-0b0d155cd00e service cinder] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:48:19.881398 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-41f0fd51-1b40-4c86-bc87-0b0d155cd00e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:48:19.979161 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0fd51-1b40-4c86-bc87-0b0d155cd00e service cinder] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:48:19.985526 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228540cb-a05c-481a-af06-68aea74fa38f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:48:19.986144 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228540cb-a05c-481a-af06-68aea74fa38f service nova] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:48:19.986420 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228540cb-a05c-481a-af06-68aea74fa38f service nova] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:48:19.986709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228540cb-a05c-481a-af06-68aea74fa38f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:48:20.002984 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-228540cb-a05c-481a-af06-68aea74fa38f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:48:20.024827 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228540cb-a05c-481a-af06-68aea74fa38f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:48:20.026710 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228540cb-a05c-481a-af06-68aea74fa38f service nova] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:48:20.048273 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1218/2441] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:48:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:48:20.106049 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb79ef9-705f-454a-8998-f6310f494895 service placement] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:48:20.108765 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-3eb79ef9-705f-454a-8998-f6310f494895 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:48:20.158413 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb79ef9-705f-454a-8998-f6310f494895 service placement] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:48:20.162162 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16bfb2e-5626-4461-abfe-bfe157f046e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:48:20.162881 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16bfb2e-5626-4461-abfe-bfe157f046e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:48:20.163027 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16bfb2e-5626-4461-abfe-bfe157f046e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:48:20.163449 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16bfb2e-5626-4461-abfe-bfe157f046e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:48:20.165087 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-d16bfb2e-5626-4461-abfe-bfe157f046e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:48:20.186173 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16bfb2e-5626-4461-abfe-bfe157f046e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:48:20.187812 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16bfb2e-5626-4461-abfe-bfe157f046e1 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:48:20.210466 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1224/2442] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:48:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:48:21.582706 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228540cb-a05c-481a-af06-68aea74fa38f service nova] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:48:21.600357 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228540cb-a05c-481a-af06-68aea74fa38f service nova] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:48:21.612009 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595ad5d-2794-4724-a608-e1647df70b10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:48:21.612835 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595ad5d-2794-4724-a608-e1647df70b10 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:48:21.613397 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595ad5d-2794-4724-a608-e1647df70b10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:48:21.613814 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595ad5d-2794-4724-a608-e1647df70b10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:48:21.616642 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-e595ad5d-2794-4724-a608-e1647df70b10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:48:21.632535 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595ad5d-2794-4724-a608-e1647df70b10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:48:21.634246 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595ad5d-2794-4724-a608-e1647df70b10 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:48:21.647913 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1219/2443] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:48:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:48:30.194715 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=15 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:48:31.642860 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=28 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:49:20.100522 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16bfb2e-5626-4461-abfe-bfe157f046e1 service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:49:20.109813 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16bfb2e-5626-4461-abfe-bfe157f046e1 service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:49:20.114339 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4dfbf-c786-4059-a356-a9c4a5edd551 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:49:20.114972 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4dfbf-c786-4059-a356-a9c4a5edd551 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:49:20.115244 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4dfbf-c786-4059-a356-a9c4a5edd551 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:49:20.115507 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4dfbf-c786-4059-a356-a9c4a5edd551 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:49:20.123205 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf4dfbf-c786-4059-a356-a9c4a5edd551 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:49:20.124622 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf4dfbf-c786-4059-a356-a9c4a5edd551 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:49:20.133219 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1225/2444] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:49:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:49:22.938643 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595ad5d-2794-4724-a608-e1647df70b10 service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:49:22.949838 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595ad5d-2794-4724-a608-e1647df70b10 service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:49:22.954345 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a180ff1-8b1c-462f-a2d3-7fdc85994178 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:49:22.955010 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a180ff1-8b1c-462f-a2d3-7fdc85994178 service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:49:22.955310 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a180ff1-8b1c-462f-a2d3-7fdc85994178 service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:49:22.955577 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a180ff1-8b1c-462f-a2d3-7fdc85994178 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:49:22.957606 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-0a180ff1-8b1c-462f-a2d3-7fdc85994178 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:49:23.032713 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a180ff1-8b1c-462f-a2d3-7fdc85994178 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:49:23.034143 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a180ff1-8b1c-462f-a2d3-7fdc85994178 service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:49:23.043475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1220/2445] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:49:22 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:49:30.129582 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=6 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:49:33.039475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=33 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:51:17.115289 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4dfbf-c786-4059-a356-a9c4a5edd551 service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:51:17.116959 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-ecf4dfbf-c786-4059-a356-a9c4a5edd551 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:51:17.196397 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4dfbf-c786-4059-a356-a9c4a5edd551 service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:51:17.202529 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326ed7f4-c188-4be9-ab11-0c98160224d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:51:17.203170 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ed7f4-c188-4be9-ab11-0c98160224d2 service nova] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:51:17.203446 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ed7f4-c188-4be9-ab11-0c98160224d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:51:17.203715 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ed7f4-c188-4be9-ab11-0c98160224d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:51:17.212193 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326ed7f4-c188-4be9-ab11-0c98160224d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:51:17.213825 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326ed7f4-c188-4be9-ab11-0c98160224d2 service nova] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:51:17.223208 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1226/2446] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:51:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:51:27.219102 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=34 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:52:21.486185 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a180ff1-8b1c-462f-a2d3-7fdc85994178 service cinder] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:52:21.488129 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-0a180ff1-8b1c-462f-a2d3-7fdc85994178 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:52:21.576252 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a180ff1-8b1c-462f-a2d3-7fdc85994178 service cinder] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:52:21.581725 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e4bb9-003c-4d64-bc4d-d20bac1195d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932d3076fb4b426d8c83d02eb60d8063', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:52:21.582332 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e4bb9-003c-4d64-bc4d-d20bac1195d1 service placement] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:52:21.582612 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e4bb9-003c-4d64-bc4d-d20bac1195d1 service placement] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:52:21.582911 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e4bb9-003c-4d64-bc4d-d20bac1195d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:52:21.584920 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-904e4bb9-003c-4d64-bc4d-d20bac1195d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:52:21.596949 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e4bb9-003c-4d64-bc4d-d20bac1195d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:52:21.598535 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e4bb9-003c-4d64-bc4d-d20bac1195d1 service placement] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:52:21.610282 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1221/2447] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:52:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:52:31.605584 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=37 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:53:19.483064 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326ed7f4-c188-4be9-ab11-0c98160224d2 service nova] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:53:19.493672 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326ed7f4-c188-4be9-ab11-0c98160224d2 service nova] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:53:19.497818 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a19923-cedd-44c3-b26a-99913c65e79f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:53:19.498425 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a19923-cedd-44c3-b26a-99913c65e79f service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:53:19.498755 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a19923-cedd-44c3-b26a-99913c65e79f service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:53:19.499062 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a19923-cedd-44c3-b26a-99913c65e79f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:53:19.510537 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a19923-cedd-44c3-b26a-99913c65e79f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:53:19.511988 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a19923-cedd-44c3-b26a-99913c65e79f service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:53:19.524863 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1227/2448] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:53:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:53:21.803692 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e4bb9-003c-4d64-bc4d-d20bac1195d1 service placement] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:53:21.813677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e4bb9-003c-4d64-bc4d-d20bac1195d1 service placement] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:53:21.817600 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0ce5e-2ba8-4d9b-a5f4-013de2326570 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:53:21.818195 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0ce5e-2ba8-4d9b-a5f4-013de2326570 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:53:21.818468 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0ce5e-2ba8-4d9b-a5f4-013de2326570 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:53:21.818779 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0ce5e-2ba8-4d9b-a5f4-013de2326570 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:53:21.829417 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e0ce5e-2ba8-4d9b-a5f4-013de2326570 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:53:21.830880 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e0ce5e-2ba8-4d9b-a5f4-013de2326570 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:53:21.841848 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1222/2449] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:53:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:53:29.518150 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=6 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:53:31.836844 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=6 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:54:22.377415 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a19923-cedd-44c3-b26a-99913c65e79f service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:54:22.397064 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a19923-cedd-44c3-b26a-99913c65e79f service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:54:22.403533 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bd058f-43a4-45a8-ab26-0a3830377996 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:54:22.404150 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd058f-43a4-45a8-ab26-0a3830377996 service nova] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:54:22.404426 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd058f-43a4-45a8-ab26-0a3830377996 service nova] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:54:22.404695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd058f-43a4-45a8-ab26-0a3830377996 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:54:22.413302 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bd058f-43a4-45a8-ab26-0a3830377996 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:54:22.414937 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bd058f-43a4-45a8-ab26-0a3830377996 service nova] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:54:22.425189 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1228/2450] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:54:22 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:54:32.420820 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=6 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:55:51.638799 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce88136-96e2-446a-9bd3-04df978a5a06 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:55:51.639528 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce88136-96e2-446a-9bd3-04df978a5a06 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:55:51.639528 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce88136-96e2-446a-9bd3-04df978a5a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:55:51.679109 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-4ce88136-96e2-446a-9bd3-04df978a5a06 None None] MFA Rules not processed for user `0e1da9439ab44ce399e72bea608c2c29`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 07:55:51.680326 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-4ce88136-96e2-446a-9bd3-04df978a5a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:55:51.699864 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1223/2451] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 07:55:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 07:55:52.036165 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bd058f-43a4-45a8-ab26-0a3830377996 service nova] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:55:52.104437 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bd058f-43a4-45a8-ab26-0a3830377996 service nova] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:55:52.109043 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b82732-5717-43e8-9d30-52ce51064b86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:55:52.109994 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b82732-5717-43e8-9d30-52ce51064b86 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:55:52.110262 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b82732-5717-43e8-9d30-52ce51064b86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:55:52.110523 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b82732-5717-43e8-9d30-52ce51064b86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:55:52.112700 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-69b82732-5717-43e8-9d30-52ce51064b86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:55:52.123353 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b82732-5717-43e8-9d30-52ce51064b86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:55:52.124815 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b82732-5717-43e8-9d30-52ce51064b86 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:55:52.134243 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1229/2452] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:55:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:55:52.489216 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce88136-96e2-446a-9bd3-04df978a5a06 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:55:52.501707 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce88136-96e2-446a-9bd3-04df978a5a06 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:55:52.507044 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4212732-92a0-4e43-99fd-75d041a26ff6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932d3076fb4b426d8c83d02eb60d8063', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:55:52.507726 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4212732-92a0-4e43-99fd-75d041a26ff6 service placement] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:55:52.508066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4212732-92a0-4e43-99fd-75d041a26ff6 service placement] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:55:52.508400 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4212732-92a0-4e43-99fd-75d041a26ff6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:55:52.510692 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-c4212732-92a0-4e43-99fd-75d041a26ff6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:55:52.522873 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4212732-92a0-4e43-99fd-75d041a26ff6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:55:52.524796 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4212732-92a0-4e43-99fd-75d041a26ff6 service placement] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:55:52.535875 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1224/2453] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:55:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:55:54.777251 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b82732-5717-43e8-9d30-52ce51064b86 service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:55:54.786064 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b82732-5717-43e8-9d30-52ce51064b86 service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:55:54.789764 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f796c1-3584-48da-8072-00cf5cb80579 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:55:54.790675 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f796c1-3584-48da-8072-00cf5cb80579 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:55:54.791044 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f796c1-3584-48da-8072-00cf5cb80579 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:55:54.791317 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f796c1-3584-48da-8072-00cf5cb80579 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:55:54.793321 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-27f796c1-3584-48da-8072-00cf5cb80579 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:55:54.805274 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f796c1-3584-48da-8072-00cf5cb80579 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:55:54.807058 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f796c1-3584-48da-8072-00cf5cb80579 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:55:54.816864 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1230/2454] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:55:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:55:55.498546 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4212732-92a0-4e43-99fd-75d041a26ff6 service placement] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:55:55.518301 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4212732-92a0-4e43-99fd-75d041a26ff6 service placement] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:55:55.524117 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a311b-29bd-4e05-ac9b-9eae8ca60e27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:55:55.524735 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a311b-29bd-4e05-ac9b-9eae8ca60e27 service nova] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:55:55.525008 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a311b-29bd-4e05-ac9b-9eae8ca60e27 service nova] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:55:55.525275 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a311b-29bd-4e05-ac9b-9eae8ca60e27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:55:55.534958 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a311b-29bd-4e05-ac9b-9eae8ca60e27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:55:55.536590 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a311b-29bd-4e05-ac9b-9eae8ca60e27 service nova] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:55:55.548250 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1225/2455] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:55:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:56:00.870013 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f796c1-3584-48da-8072-00cf5cb80579 service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:56:00.886273 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f796c1-3584-48da-8072-00cf5cb80579 service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:56:00.892454 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b3071-0f81-41e2-aaba-f17c2d7a7ec5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:56:00.893070 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b3071-0f81-41e2-aaba-f17c2d7a7ec5 service nova] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:56:00.893348 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b3071-0f81-41e2-aaba-f17c2d7a7ec5 service nova] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:56:00.893636 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b3071-0f81-41e2-aaba-f17c2d7a7ec5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:56:00.895472 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-a10b3071-0f81-41e2-aaba-f17c2d7a7ec5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:56:00.908028 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b3071-0f81-41e2-aaba-f17c2d7a7ec5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:56:00.909618 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b3071-0f81-41e2-aaba-f17c2d7a7ec5 service nova] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:56:00.928578 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1231/2456] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:56:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:56:05.543241 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=26 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:56:10.919531 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=38 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:56:22.477738 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a311b-29bd-4e05-ac9b-9eae8ca60e27 service nova] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:56:22.479505 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-df1a311b-29bd-4e05-ac9b-9eae8ca60e27 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:56:22.562874 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a311b-29bd-4e05-ac9b-9eae8ca60e27 service nova] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:56:22.567172 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb39e9d-5b45-4bee-a839-c99972af33fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:56:22.567781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb39e9d-5b45-4bee-a839-c99972af33fe service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:56:22.568055 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb39e9d-5b45-4bee-a839-c99972af33fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:56:22.568326 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb39e9d-5b45-4bee-a839-c99972af33fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:56:22.570250 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-eeb39e9d-5b45-4bee-a839-c99972af33fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:56:22.657468 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb39e9d-5b45-4bee-a839-c99972af33fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:56:22.658928 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb39e9d-5b45-4bee-a839-c99972af33fe service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:56:22.682696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1226/2457] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:56:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:56:32.678456 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=70 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:57:26.546469 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b3071-0f81-41e2-aaba-f17c2d7a7ec5 service nova] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:57:26.549135 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-a10b3071-0f81-41e2-aaba-f17c2d7a7ec5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 07:57:26.624029 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b3071-0f81-41e2-aaba-f17c2d7a7ec5 service nova] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:57:26.629680 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c4944c-46d7-402a-8eca-7f02af97aa2a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932d3076fb4b426d8c83d02eb60d8063', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:57:26.630291 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4944c-46d7-402a-8eca-7f02af97aa2a service placement] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:57:26.630566 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4944c-46d7-402a-8eca-7f02af97aa2a service placement] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:57:26.630869 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4944c-46d7-402a-8eca-7f02af97aa2a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:57:26.639447 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c4944c-46d7-402a-8eca-7f02af97aa2a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:57:26.641119 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c4944c-46d7-402a-8eca-7f02af97aa2a service placement] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:57:26.651326 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1232/2458] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:57:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:57:36.647225 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=28 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 07:59:22.621707 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb39e9d-5b45-4bee-a839-c99972af33fe service cinder] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 07:59:22.643180 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb39e9d-5b45-4bee-a839-c99972af33fe service cinder] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 07:59:22.647655 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11ed67-f7ff-4046-9404-420bf10cbdf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 07:59:22.648294 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11ed67-f7ff-4046-9404-420bf10cbdf8 service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 07:59:22.648563 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11ed67-f7ff-4046-9404-420bf10cbdf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 07:59:22.648828 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11ed67-f7ff-4046-9404-420bf10cbdf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 07:59:22.658043 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11ed67-f7ff-4046-9404-420bf10cbdf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 07:59:22.659520 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11ed67-f7ff-4046-9404-420bf10cbdf8 service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 07:59:22.669264 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1227/2459] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 07:59:22 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 07:59:32.664954 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=11 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:00:23.003266 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c4944c-46d7-402a-8eca-7f02af97aa2a service placement] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:00:23.004937 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-41c4944c-46d7-402a-8eca-7f02af97aa2a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:00:23.054377 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c4944c-46d7-402a-8eca-7f02af97aa2a service placement] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:00:23.058703 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479a087-b891-4570-982b-cb0ab951d0df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:00:23.059303 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479a087-b891-4570-982b-cb0ab951d0df service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:00:23.059572 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479a087-b891-4570-982b-cb0ab951d0df service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:00:23.059837 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479a087-b891-4570-982b-cb0ab951d0df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:00:23.061578 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-6479a087-b891-4570-982b-cb0ab951d0df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:00:23.149956 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6479a087-b891-4570-982b-cb0ab951d0df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:00:23.151785 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6479a087-b891-4570-982b-cb0ab951d0df service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:00:23.172544 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1233/2460] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:00:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:00:26.146153 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11ed67-f7ff-4046-9404-420bf10cbdf8 service cinder] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:00:26.159229 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11ed67-f7ff-4046-9404-420bf10cbdf8 service cinder] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:00:26.164664 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c42a2-991b-4919-ae92-baa42f6386c3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932d3076fb4b426d8c83d02eb60d8063', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:00:26.165279 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c42a2-991b-4919-ae92-baa42f6386c3 service placement] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:00:26.165551 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c42a2-991b-4919-ae92-baa42f6386c3 service placement] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:00:26.165817 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c42a2-991b-4919-ae92-baa42f6386c3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:00:26.167774 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-051c42a2-991b-4919-ae92-baa42f6386c3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:00:26.181855 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051c42a2-991b-4919-ae92-baa42f6386c3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:00:26.183677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051c42a2-991b-4919-ae92-baa42f6386c3 service placement] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:00:26.198845 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1228/2461] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:00:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:00:29.196663 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479a087-b891-4570-982b-cb0ab951d0df service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:00:29.198276 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-6479a087-b891-4570-982b-cb0ab951d0df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:00:29.219357 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479a087-b891-4570-982b-cb0ab951d0df service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:00:29.224387 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445c96d-3b2f-4c18-b1c6-5e10645b2376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:00:29.225108 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445c96d-3b2f-4c18-b1c6-5e10645b2376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:00:29.225683 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445c96d-3b2f-4c18-b1c6-5e10645b2376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:00:29.226341 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445c96d-3b2f-4c18-b1c6-5e10645b2376 admin admin] PATH_INFO: `/v3/users/a47b81144eba4283aac610c600e73ed3` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:00:29.228601 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2445c96d-3b2f-4c18-b1c6-5e10645b2376 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a47b81144eba4283aac610c600e73ed3)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:00:29.231405 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2445c96d-3b2f-4c18-b1c6-5e10645b2376 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:00:29.294905 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-2445c96d-3b2f-4c18-b1c6-5e10645b2376 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a47b81144eba4283aac610c600e73ed3'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:00:29.303641 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-2445c96d-3b2f-4c18-b1c6-5e10645b2376 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a47b81144eba4283aac610c600e73ed3'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:00:29.304487 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-2445c96d-3b2f-4c18-b1c6-5e10645b2376 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a47b81144eba4283aac610c600e73ed3'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:00:29.319461 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-2445c96d-3b2f-4c18-b1c6-5e10645b2376 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a47b81144eba4283aac610c600e73ed3'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:00:29.331222 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-2445c96d-3b2f-4c18-b1c6-5e10645b2376 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a47b81144eba4283aac610c600e73ed3'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:00:29.338585 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1234/2462] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 08:00:29 2022] DELETE /identity/v3/users/a47b81144eba4283aac610c600e73ed3 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:00:29.537051 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c42a2-991b-4919-ae92-baa42f6386c3 service placement] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:00:29.549463 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c42a2-991b-4919-ae92-baa42f6386c3 service placement] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:00:29.608763 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276fe6b-b5d7-4140-b386-e69513c8a87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:00:29.609400 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276fe6b-b5d7-4140-b386-e69513c8a87b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:00:29.609674 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276fe6b-b5d7-4140-b386-e69513c8a87b admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:00:29.609939 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276fe6b-b5d7-4140-b386-e69513c8a87b admin admin] PATH_INFO: `/v3/projects/e492702cf15848cda2fa93e1db845c25` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:00:29.611074 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276fe6b-b5d7-4140-b386-e69513c8a87b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e492702cf15848cda2fa93e1db845c25)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:00:29.612974 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276fe6b-b5d7-4140-b386-e69513c8a87b admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:00:29.666670 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c276fe6b-b5d7-4140-b386-e69513c8a87b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e492702cf15848cda2fa93e1db845c25', 'request_id': 'req-c276fe6b-b5d7-4140-b386-e69513c8a87b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:00:29.673112 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c276fe6b-b5d7-4140-b386-e69513c8a87b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e492702cf15848cda2fa93e1db845c25', 'request_id': 'req-c276fe6b-b5d7-4140-b386-e69513c8a87b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:00:29.678798 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c276fe6b-b5d7-4140-b386-e69513c8a87b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e492702cf15848cda2fa93e1db845c25', 'request_id': 'req-c276fe6b-b5d7-4140-b386-e69513c8a87b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:00:29.694431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c276fe6b-b5d7-4140-b386-e69513c8a87b admin admin] The token cache is being invalidate because project e492702cf15848cda2fa93e1db845c25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 08:00:29.694830 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c276fe6b-b5d7-4140-b386-e69513c8a87b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:00:29.695526 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c276fe6b-b5d7-4140-b386-e69513c8a87b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:00:29.697105 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1229/2463] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 08:00:29 2022] DELETE /identity/v3/projects/e492702cf15848cda2fa93e1db845c25 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:00:29.701845 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445c96d-3b2f-4c18-b1c6-5e10645b2376 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:00:29.703355 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-2445c96d-3b2f-4c18-b1c6-5e10645b2376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:00:29.717015 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445c96d-3b2f-4c18-b1c6-5e10645b2376 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:00:29.722023 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:00:29.778106 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce936cf6-e79a-4863-85e7-c9e21f7c88d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:00:29.778767 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce936cf6-e79a-4863-85e7-c9e21f7c88d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:00:29.779035 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce936cf6-e79a-4863-85e7-c9e21f7c88d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:00:29.779306 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce936cf6-e79a-4863-85e7-c9e21f7c88d5 admin admin] PATH_INFO: `/v3/users/d101c78015b842f99ea0662f60aa7283` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:00:29.793293 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce936cf6-e79a-4863-85e7-c9e21f7c88d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d101c78015b842f99ea0662f60aa7283)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:00:29.795256 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce936cf6-e79a-4863-85e7-c9e21f7c88d5 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:00:29.853646 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-ce936cf6-e79a-4863-85e7-c9e21f7c88d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd101c78015b842f99ea0662f60aa7283'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:00:29.861043 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-ce936cf6-e79a-4863-85e7-c9e21f7c88d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd101c78015b842f99ea0662f60aa7283'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:00:29.861704 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-ce936cf6-e79a-4863-85e7-c9e21f7c88d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd101c78015b842f99ea0662f60aa7283'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:00:29.874220 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-ce936cf6-e79a-4863-85e7-c9e21f7c88d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd101c78015b842f99ea0662f60aa7283'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:00:29.883256 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-ce936cf6-e79a-4863-85e7-c9e21f7c88d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd101c78015b842f99ea0662f60aa7283'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:00:29.889369 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1235/2464] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 08:00:29 2022] DELETE /identity/v3/users/d101c78015b842f99ea0662f60aa7283 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:00:30.073020 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276fe6b-b5d7-4140-b386-e69513c8a87b admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:00:30.074087 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-c276fe6b-b5d7-4140-b386-e69513c8a87b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:00:30.087811 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276fe6b-b5d7-4140-b386-e69513c8a87b admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:00:30.146331 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd35d1e-9a1b-4fca-90dc-51b1f5fba33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:00:30.146969 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd35d1e-9a1b-4fca-90dc-51b1f5fba33e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:00:30.147245 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd35d1e-9a1b-4fca-90dc-51b1f5fba33e admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:00:30.147604 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd35d1e-9a1b-4fca-90dc-51b1f5fba33e admin admin] PATH_INFO: `/v3/projects/e4253bbcacfe4775940efe5160a90309` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:00:30.159339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd35d1e-9a1b-4fca-90dc-51b1f5fba33e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4253bbcacfe4775940efe5160a90309)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:00:30.161268 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd35d1e-9a1b-4fca-90dc-51b1f5fba33e admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:00:30.172932 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:00:30.214907 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-3dd35d1e-9a1b-4fca-90dc-51b1f5fba33e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4253bbcacfe4775940efe5160a90309', 'request_id': 'req-3dd35d1e-9a1b-4fca-90dc-51b1f5fba33e'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:00:30.221624 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-3dd35d1e-9a1b-4fca-90dc-51b1f5fba33e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4253bbcacfe4775940efe5160a90309', 'request_id': 'req-3dd35d1e-9a1b-4fca-90dc-51b1f5fba33e'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:00:30.228064 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-3dd35d1e-9a1b-4fca-90dc-51b1f5fba33e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4253bbcacfe4775940efe5160a90309', 'request_id': 'req-3dd35d1e-9a1b-4fca-90dc-51b1f5fba33e'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:00:30.242939 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-3dd35d1e-9a1b-4fca-90dc-51b1f5fba33e admin admin] The token cache is being invalidate because project e4253bbcacfe4775940efe5160a90309 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 08:00:30.243220 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-3dd35d1e-9a1b-4fca-90dc-51b1f5fba33e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:00:30.243839 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-3dd35d1e-9a1b-4fca-90dc-51b1f5fba33e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:00:30.245327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1230/2465] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 08:00:30 2022] DELETE /identity/v3/projects/e4253bbcacfe4775940efe5160a90309 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:00:39.886257 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=54,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:00:40.240150 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=20,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:02:28.446494 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce936cf6-e79a-4863-85e7-c9e21f7c88d5 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:02:28.448157 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-ce936cf6-e79a-4863-85e7-c9e21f7c88d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:02:28.534676 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce936cf6-e79a-4863-85e7-c9e21f7c88d5 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:02:28.605259 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4825da-4070-4f1f-bfe5-096a7c330b2b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932d3076fb4b426d8c83d02eb60d8063', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:02:28.605886 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4825da-4070-4f1f-bfe5-096a7c330b2b service placement] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:02:28.606196 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4825da-4070-4f1f-bfe5-096a7c330b2b service placement] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:02:28.606467 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4825da-4070-4f1f-bfe5-096a7c330b2b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:02:28.608368 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-bf4825da-4070-4f1f-bfe5-096a7c330b2b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:02:28.620648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4825da-4070-4f1f-bfe5-096a7c330b2b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:02:28.622263 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4825da-4070-4f1f-bfe5-096a7c330b2b service placement] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:02:28.633328 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1236/2466] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:02:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:02:38.628733 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=63 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:03:52.130229 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd35d1e-9a1b-4fca-90dc-51b1f5fba33e admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:03:52.131677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-3dd35d1e-9a1b-4fca-90dc-51b1f5fba33e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:03:52.189823 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd35d1e-9a1b-4fca-90dc-51b1f5fba33e admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:03:52.193784 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6c178-35ed-404d-8992-604de86bf6b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:03:52.194389 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6c178-35ed-404d-8992-604de86bf6b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:03:52.194634 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6c178-35ed-404d-8992-604de86bf6b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:03:52.194963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6c178-35ed-404d-8992-604de86bf6b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:03:52.196699 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-90a6c178-35ed-404d-8992-604de86bf6b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:03:52.261218 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a6c178-35ed-404d-8992-604de86bf6b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:03:52.262568 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a6c178-35ed-404d-8992-604de86bf6b6 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:03:52.273006 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1231/2467] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:03:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:03:55.542846 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4825da-4070-4f1f-bfe5-096a7c330b2b service placement] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:03:55.544350 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-bf4825da-4070-4f1f-bfe5-096a7c330b2b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:03:55.559969 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4825da-4070-4f1f-bfe5-096a7c330b2b service placement] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:03:55.564737 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f0669-991b-4672-a37a-5f46e7787638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:03:55.565362 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f0669-991b-4672-a37a-5f46e7787638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:03:55.565639 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f0669-991b-4672-a37a-5f46e7787638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:03:55.565914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f0669-991b-4672-a37a-5f46e7787638 admin admin] PATH_INFO: `/v3/users/0e1da9439ab44ce399e72bea608c2c29` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:03:55.579856 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f0669-991b-4672-a37a-5f46e7787638 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e1da9439ab44ce399e72bea608c2c29)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:03:55.581974 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f0669-991b-4672-a37a-5f46e7787638 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:03:55.634334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-b00f0669-991b-4672-a37a-5f46e7787638 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e1da9439ab44ce399e72bea608c2c29'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:03:55.641302 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-b00f0669-991b-4672-a37a-5f46e7787638 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e1da9439ab44ce399e72bea608c2c29'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:03:55.642159 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-b00f0669-991b-4672-a37a-5f46e7787638 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e1da9439ab44ce399e72bea608c2c29'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:03:55.655892 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-b00f0669-991b-4672-a37a-5f46e7787638 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e1da9439ab44ce399e72bea608c2c29'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:03:55.664613 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-b00f0669-991b-4672-a37a-5f46e7787638 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e1da9439ab44ce399e72bea608c2c29'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:03:55.670703 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1237/2468] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 08:03:55 2022] DELETE /identity/v3/users/0e1da9439ab44ce399e72bea608c2c29 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:03:55.868206 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6c178-35ed-404d-8992-604de86bf6b6 service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:03:55.880547 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6c178-35ed-404d-8992-604de86bf6b6 service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:03:55.941395 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423ea6c-d2f9-4d33-898f-f6fdeb155854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:03:55.942057 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423ea6c-d2f9-4d33-898f-f6fdeb155854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:03:55.942333 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423ea6c-d2f9-4d33-898f-f6fdeb155854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:03:55.942585 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423ea6c-d2f9-4d33-898f-f6fdeb155854 admin admin] PATH_INFO: `/v3/projects/f1d002fd08c941a798ee709ace20f420` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:03:55.943755 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e423ea6c-d2f9-4d33-898f-f6fdeb155854 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1d002fd08c941a798ee709ace20f420)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:03:55.945658 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e423ea6c-d2f9-4d33-898f-f6fdeb155854 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:03:55.995890 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-e423ea6c-d2f9-4d33-898f-f6fdeb155854 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1d002fd08c941a798ee709ace20f420', 'request_id': 'req-e423ea6c-d2f9-4d33-898f-f6fdeb155854'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:03:55.997652 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:03:56.003893 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-e423ea6c-d2f9-4d33-898f-f6fdeb155854 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1d002fd08c941a798ee709ace20f420', 'request_id': 'req-e423ea6c-d2f9-4d33-898f-f6fdeb155854'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:03:56.010276 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-e423ea6c-d2f9-4d33-898f-f6fdeb155854 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1d002fd08c941a798ee709ace20f420', 'request_id': 'req-e423ea6c-d2f9-4d33-898f-f6fdeb155854'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:03:56.025616 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-e423ea6c-d2f9-4d33-898f-f6fdeb155854 admin admin] The token cache is being invalidate because project f1d002fd08c941a798ee709ace20f420 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 08:03:56.025904 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-e423ea6c-d2f9-4d33-898f-f6fdeb155854 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:03:56.026515 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-e423ea6c-d2f9-4d33-898f-f6fdeb155854 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:03:56.028072 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1232/2469] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 08:03:55 2022] DELETE /identity/v3/projects/f1d002fd08c941a798ee709ace20f420 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:03:56.034154 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ba40d-b1f0-45ca-a2b8-12483decd4f5 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:03:56.034438 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ba40d-b1f0-45ca-a2b8-12483decd4f5 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:03:56.034738 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ba40d-b1f0-45ca-a2b8-12483decd4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:03:56.094752 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-924ba40d-b1f0-45ca-a2b8-12483decd4f5 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:03:56.096185 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-924ba40d-b1f0-45ca-a2b8-12483decd4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:03:56.164951 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1238/2470] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 08:03:56.173139 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63c4bf-2981-4c6b-a275-efef64be2c0e None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:03:56.173466 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63c4bf-2981-4c6b-a275-efef64be2c0e None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:03:56.174040 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63c4bf-2981-4c6b-a275-efef64be2c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:03:56.205699 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-3d63c4bf-2981-4c6b-a275-efef64be2c0e None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:03:56.206938 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-3d63c4bf-2981-4c6b-a275-efef64be2c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:03:56.216288 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1233/2471] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 08:03:56.220804 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ba40d-b1f0-45ca-a2b8-12483decd4f5 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:03:56.231208 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ba40d-b1f0-45ca-a2b8-12483decd4f5 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:03:56.236256 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149301e-c6d5-4479-b1a9-e7ffa39ddb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:03:56.236856 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149301e-c6d5-4479-b1a9-e7ffa39ddb12 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:03:56.237125 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149301e-c6d5-4479-b1a9-e7ffa39ddb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:03:56.237406 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149301e-c6d5-4479-b1a9-e7ffa39ddb12 admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:03:56.238046 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149301e-c6d5-4479-b1a9-e7ffa39ddb12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 08:03:56.238344 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149301e-c6d5-4479-b1a9-e7ffa39ddb12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:03:56.240015 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149301e-c6d5-4479-b1a9-e7ffa39ddb12 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:03:56.252155 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1239/2472] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 08:03:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 08:03:56.256514 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63c4bf-2981-4c6b-a275-efef64be2c0e None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:03:56.265942 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63c4bf-2981-4c6b-a275-efef64be2c0e None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:03:56.271580 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cae70b-52c3-4a0e-a97c-1c407c777ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:03:56.272187 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cae70b-52c3-4a0e-a97c-1c407c777ef8 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:03:56.272458 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cae70b-52c3-4a0e-a97c-1c407c777ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:03:56.272730 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cae70b-52c3-4a0e-a97c-1c407c777ef8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:03:56.273356 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cae70b-52c3-4a0e-a97c-1c407c777ef8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:03:56.275308 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cae70b-52c3-4a0e-a97c-1c407c777ef8 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:03:56.316518 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1234/2473] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 08:03:56 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 08:03:56.320892 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149301e-c6d5-4479-b1a9-e7ffa39ddb12 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:03:56.336402 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149301e-c6d5-4479-b1a9-e7ffa39ddb12 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:03:56.350954 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:03:56.400992 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d26016a-9166-4b3a-83d0-8e90dbf04314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:03:56.401603 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26016a-9166-4b3a-83d0-8e90dbf04314 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:03:56.401906 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26016a-9166-4b3a-83d0-8e90dbf04314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:03:56.402170 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26016a-9166-4b3a-83d0-8e90dbf04314 admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:03:56.402821 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d26016a-9166-4b3a-83d0-8e90dbf04314 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:03:56.404746 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d26016a-9166-4b3a-83d0-8e90dbf04314 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:03:56.429241 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1240/2474] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 08:03:56 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 08:03:56.433557 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cae70b-52c3-4a0e-a97c-1c407c777ef8 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:03:56.444795 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cae70b-52c3-4a0e-a97c-1c407c777ef8 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:03:56.450516 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0768e8-6124-4d4b-89e3-6f72a302840f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:03:56.451124 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0768e8-6124-4d4b-89e3-6f72a302840f admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:03:56.451399 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0768e8-6124-4d4b-89e3-6f72a302840f admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:03:56.451721 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0768e8-6124-4d4b-89e3-6f72a302840f admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:03:56.452595 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0768e8-6124-4d4b-89e3-6f72a302840f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:03:56.454201 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0768e8-6124-4d4b-89e3-6f72a302840f admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:03:56.464842 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1235/2475] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 08:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 08:03:56.468917 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d26016a-9166-4b3a-83d0-8e90dbf04314 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:03:56.480004 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d26016a-9166-4b3a-83d0-8e90dbf04314 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:03:56.485451 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937557da-6dc9-470e-875e-5a1ed7dacb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:03:56.486172 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937557da-6dc9-470e-875e-5a1ed7dacb25 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:03:56.486622 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937557da-6dc9-470e-875e-5a1ed7dacb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:03:56.486767 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937557da-6dc9-470e-875e-5a1ed7dacb25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:03:56.487344 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937557da-6dc9-470e-875e-5a1ed7dacb25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:03:56.488942 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937557da-6dc9-470e-875e-5a1ed7dacb25 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:03:56.499808 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1241/2476] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 08:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 08:03:56.504206 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0768e8-6124-4d4b-89e3-6f72a302840f admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:03:56.514454 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0768e8-6124-4d4b-89e3-6f72a302840f admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:03:56.519917 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16da1a-d29a-4150-ae00-d47e301fe858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:03:56.520571 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16da1a-d29a-4150-ae00-d47e301fe858 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:03:56.520821 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16da1a-d29a-4150-ae00-d47e301fe858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:03:56.521114 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16da1a-d29a-4150-ae00-d47e301fe858 admin admin] PATH_INFO: `/v3/projects/38d7aef5eecc465fae2a5158ec6aa6e2/users/8972201328be4283af6873ec700dcf42/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:03:56.536497 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a16da1a-d29a-4150-ae00-d47e301fe858 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38d7aef5eecc465fae2a5158ec6aa6e2, user_id=8972201328be4283af6873ec700dcf42, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:03:56.539164 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a16da1a-d29a-4150-ae00-d47e301fe858 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:03:56.555390 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1236/2477] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 08:03:56 2022] PUT /identity/v3/projects/38d7aef5eecc465fae2a5158ec6aa6e2/users/8972201328be4283af6873ec700dcf42/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:03:56.562214 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cab0c4-13ab-46c5-af06-c93c5e377775 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:03:56.562718 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cab0c4-13ab-46c5-af06-c93c5e377775 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:03:56.562824 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cab0c4-13ab-46c5-af06-c93c5e377775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:03:56.582134 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-20cab0c4-13ab-46c5-af06-c93c5e377775 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:03:56.583377 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-20cab0c4-13ab-46c5-af06-c93c5e377775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:03:56.650530 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1242/2478] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 08:03:56.662793 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16da1a-d29a-4150-ae00-d47e301fe858 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:03:56.665894 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-8a16da1a-d29a-4150-ae00-d47e301fe858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:03:56.735615 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16da1a-d29a-4150-ae00-d47e301fe858 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:03:56.740161 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87368ca8-3153-4682-ac66-44fdc76e4b32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:03:56.740793 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87368ca8-3153-4682-ac66-44fdc76e4b32 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:03:56.741063 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87368ca8-3153-4682-ac66-44fdc76e4b32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:03:56.741327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87368ca8-3153-4682-ac66-44fdc76e4b32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:03:56.750804 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87368ca8-3153-4682-ac66-44fdc76e4b32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:03:56.752571 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87368ca8-3153-4682-ac66-44fdc76e4b32 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:03:56.762521 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1237/2479] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:03:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:04:00.935250 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2d1ce-7c52-443b-b861-dd616db980f2 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:04:00.935773 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2d1ce-7c52-443b-b861-dd616db980f2 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:04:00.936042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2d1ce-7c52-443b-b861-dd616db980f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:04:00.966798 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-67e2d1ce-7c52-443b-b861-dd616db980f2 None None] MFA Rules not processed for user `8972201328be4283af6873ec700dcf42`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:04:00.968233 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-67e2d1ce-7c52-443b-b861-dd616db980f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:04:01.047752 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1243/2480] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:04:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 08:04:01.062813 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87368ca8-3153-4682-ac66-44fdc76e4b32 service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:04:01.066176 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-87368ca8-3153-4682-ac66-44fdc76e4b32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:04:01.141493 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:04:01.178425 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87368ca8-3153-4682-ac66-44fdc76e4b32 service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:04:01.186801 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a0fe6-aeca-437d-837c-44f6257c9b0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:04:01.187428 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a0fe6-aeca-437d-837c-44f6257c9b0a service nova] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:04:01.187701 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a0fe6-aeca-437d-837c-44f6257c9b0a service nova] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:04:01.187985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a0fe6-aeca-437d-837c-44f6257c9b0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:04:01.197772 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232a0fe6-aeca-437d-837c-44f6257c9b0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:04:01.199538 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232a0fe6-aeca-437d-837c-44f6257c9b0a service nova] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:04:01.209486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1238/2481] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:04:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:04:02.547895 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74738c-1537-41df-9b0c-9711c522cd0b None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:04:02.548429 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74738c-1537-41df-9b0c-9711c522cd0b None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:04:02.548641 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74738c-1537-41df-9b0c-9711c522cd0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:04:02.568851 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-8d74738c-1537-41df-9b0c-9711c522cd0b None None] MFA Rules not processed for user `8972201328be4283af6873ec700dcf42`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:04:02.570071 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-8d74738c-1537-41df-9b0c-9711c522cd0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:04:02.577667 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1244/2482] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:04:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 08:04:02.619101 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a0fe6-aeca-437d-837c-44f6257c9b0a service nova] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:04:02.620605 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-232a0fe6-aeca-437d-837c-44f6257c9b0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:04:02.707728 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a0fe6-aeca-437d-837c-44f6257c9b0a service nova] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:04:02.714180 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596bae51-f673-41ab-96f8-20092698ec3c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a051ca8154540ad93db6d028b8323a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:04:02.715067 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596bae51-f673-41ab-96f8-20092698ec3c service glance] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:04:02.715420 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596bae51-f673-41ab-96f8-20092698ec3c service glance] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:04:02.715709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596bae51-f673-41ab-96f8-20092698ec3c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:04:02.717753 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-596bae51-f673-41ab-96f8-20092698ec3c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:04:02.735335 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596bae51-f673-41ab-96f8-20092698ec3c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:04:02.737536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596bae51-f673-41ab-96f8-20092698ec3c service glance] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:04:02.749384 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1239/2483] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:04:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:04:03.062497 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d74738c-1537-41df-9b0c-9711c522cd0b None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:04:03.073422 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d74738c-1537-41df-9b0c-9711c522cd0b None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:04:03.077335 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9402de61-1234-45d1-a374-ee8a9092232b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:04:03.078000 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402de61-1234-45d1-a374-ee8a9092232b service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:04:03.078336 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402de61-1234-45d1-a374-ee8a9092232b service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:04:03.078551 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402de61-1234-45d1-a374-ee8a9092232b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:04:03.080498 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-9402de61-1234-45d1-a374-ee8a9092232b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:04:03.091870 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9402de61-1234-45d1-a374-ee8a9092232b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:04:03.093648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9402de61-1234-45d1-a374-ee8a9092232b service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:04:03.104482 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1245/2484] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:04:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:04:03.452642 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596bae51-f673-41ab-96f8-20092698ec3c service glance] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:04:03.456146 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-596bae51-f673-41ab-96f8-20092698ec3c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:04:03.461131 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9402de61-1234-45d1-a374-ee8a9092232b service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:04:03.462544 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-9402de61-1234-45d1-a374-ee8a9092232b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:04:03.468889 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596bae51-f673-41ab-96f8-20092698ec3c service glance] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:04:03.473230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2f636-eedc-46a2-8187-28a9ae5e14d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:04:03.473924 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2f636-eedc-46a2-8187-28a9ae5e14d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:04:03.474219 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2f636-eedc-46a2-8187-28a9ae5e14d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:04:03.474497 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2f636-eedc-46a2-8187-28a9ae5e14d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:04:03.476656 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-4ac2f636-eedc-46a2-8187-28a9ae5e14d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:04:03.477153 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:04:03.496888 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2f636-eedc-46a2-8187-28a9ae5e14d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:04:03.498730 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2f636-eedc-46a2-8187-28a9ae5e14d2 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:04:03.511305 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1240/2485] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:04:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:04:03.553810 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9402de61-1234-45d1-a374-ee8a9092232b service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:04:03.559382 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c1f9f-9fa0-4c31-938b-76c6f5f0fa98 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932d3076fb4b426d8c83d02eb60d8063', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:04:03.560002 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c1f9f-9fa0-4c31-938b-76c6f5f0fa98 service placement] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:04:03.560279 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c1f9f-9fa0-4c31-938b-76c6f5f0fa98 service placement] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:04:03.560673 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c1f9f-9fa0-4c31-938b-76c6f5f0fa98 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:04:03.562648 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-342c1f9f-9fa0-4c31-938b-76c6f5f0fa98 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:04:03.575427 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c1f9f-9fa0-4c31-938b-76c6f5f0fa98 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:04:03.577028 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c1f9f-9fa0-4c31-938b-76c6f5f0fa98 service placement] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:04:03.590696 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1246/2486] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:04:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:04:05.870078 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2f636-eedc-46a2-8187-28a9ae5e14d2 service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:04:05.885692 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2f636-eedc-46a2-8187-28a9ae5e14d2 service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:04:05.894846 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c7154-d5c5-4e62-a913-cc08b761d7a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:04:05.895480 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c7154-d5c5-4e62-a913-cc08b761d7a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:04:05.895780 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c7154-d5c5-4e62-a913-cc08b761d7a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:04:05.896049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c7154-d5c5-4e62-a913-cc08b761d7a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:04:05.897803 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-061c7154-d5c5-4e62-a913-cc08b761d7a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:04:05.907641 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061c7154-d5c5-4e62-a913-cc08b761d7a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:04:05.909321 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061c7154-d5c5-4e62-a913-cc08b761d7a2 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:04:05.917310 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1241/2487] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:04:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:04:06.568351 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c1f9f-9fa0-4c31-938b-76c6f5f0fa98 service placement] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:04:06.569917 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-342c1f9f-9fa0-4c31-938b-76c6f5f0fa98 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:04:06.587219 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c1f9f-9fa0-4c31-938b-76c6f5f0fa98 service placement] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:04:06.593314 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e31dfd-fa8e-43a3-84f3-bffd295ec9dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:04:06.594017 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e31dfd-fa8e-43a3-84f3-bffd295ec9dd service nova] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:04:06.594345 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e31dfd-fa8e-43a3-84f3-bffd295ec9dd service nova] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:04:06.594705 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e31dfd-fa8e-43a3-84f3-bffd295ec9dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:04:06.597073 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-80e31dfd-fa8e-43a3-84f3-bffd295ec9dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:04:06.611895 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e31dfd-fa8e-43a3-84f3-bffd295ec9dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:04:06.613660 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e31dfd-fa8e-43a3-84f3-bffd295ec9dd service nova] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:04:06.631412 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1247/2488] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:04:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:04:13.059659 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c7154-d5c5-4e62-a913-cc08b761d7a2 service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:04:13.075560 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c7154-d5c5-4e62-a913-cc08b761d7a2 service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:04:13.081636 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6988b261-aad9-4e7e-83ef-c84d37a146aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:04:13.082230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988b261-aad9-4e7e-83ef-c84d37a146aa service nova] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:04:13.082515 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988b261-aad9-4e7e-83ef-c84d37a146aa service nova] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:04:13.082820 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988b261-aad9-4e7e-83ef-c84d37a146aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:04:13.084787 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-6988b261-aad9-4e7e-83ef-c84d37a146aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:04:13.097424 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6988b261-aad9-4e7e-83ef-c84d37a146aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:04:13.099191 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6988b261-aad9-4e7e-83ef-c84d37a146aa service nova] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:04:13.110258 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1242/2489] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:04:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:04:16.626334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=35 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:04:23.105949 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=67 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:07:32.441713 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e31dfd-fa8e-43a3-84f3-bffd295ec9dd service nova] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:07:32.443614 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-80e31dfd-fa8e-43a3-84f3-bffd295ec9dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:07:32.501676 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e31dfd-fa8e-43a3-84f3-bffd295ec9dd service nova] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:07:32.507178 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6d08e-ca55-44ee-8814-69d6e89e48b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932d3076fb4b426d8c83d02eb60d8063', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:07:32.507793 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6d08e-ca55-44ee-8814-69d6e89e48b9 service placement] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:07:32.508072 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6d08e-ca55-44ee-8814-69d6e89e48b9 service placement] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:07:32.508346 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6d08e-ca55-44ee-8814-69d6e89e48b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:07:32.510193 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-16b6d08e-ca55-44ee-8814-69d6e89e48b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:07:32.524134 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b6d08e-ca55-44ee-8814-69d6e89e48b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:07:32.525726 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b6d08e-ca55-44ee-8814-69d6e89e48b9 service placement] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:07:32.537513 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1248/2490] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:07:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:07:42.532558 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=23 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:08:59.256796 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a70b3b-a6a2-4cce-8aeb-45ab3a7e70bf None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:08:59.257363 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a70b3b-a6a2-4cce-8aeb-45ab3a7e70bf None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:08:59.257482 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a70b3b-a6a2-4cce-8aeb-45ab3a7e70bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:08:59.277734 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-39a70b3b-a6a2-4cce-8aeb-45ab3a7e70bf None None] MFA Rules not processed for user `8972201328be4283af6873ec700dcf42`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:08:59.278966 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-39a70b3b-a6a2-4cce-8aeb-45ab3a7e70bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:08:59.286498 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1243/2491] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:08:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 08:08:59.617604 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6d08e-ca55-44ee-8814-69d6e89e48b9 service placement] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:08:59.628130 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6d08e-ca55-44ee-8814-69d6e89e48b9 service placement] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:08:59.631944 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb1bb2-27c9-409c-8a35-95d6d426462f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:08:59.632540 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb1bb2-27c9-409c-8a35-95d6d426462f service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:08:59.632814 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb1bb2-27c9-409c-8a35-95d6d426462f service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:08:59.633082 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb1bb2-27c9-409c-8a35-95d6d426462f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:08:59.635333 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-83fb1bb2-27c9-409c-8a35-95d6d426462f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:08:59.647186 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb1bb2-27c9-409c-8a35-95d6d426462f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:08:59.649311 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb1bb2-27c9-409c-8a35-95d6d426462f service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:08:59.662363 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1249/2492] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:08:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:09:01.101229 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a70b3b-a6a2-4cce-8aeb-45ab3a7e70bf None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:09:01.116846 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a70b3b-a6a2-4cce-8aeb-45ab3a7e70bf None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:09:01.124509 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6225e2-085e-4734-a136-fa034921e73b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:09:01.125109 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6225e2-085e-4734-a136-fa034921e73b service nova] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:09:01.125381 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6225e2-085e-4734-a136-fa034921e73b service nova] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:09:01.125647 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6225e2-085e-4734-a136-fa034921e73b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:09:01.133755 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6225e2-085e-4734-a136-fa034921e73b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:09:01.135374 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6225e2-085e-4734-a136-fa034921e73b service nova] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:09:01.144311 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1244/2493] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:09:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:09:03.498636 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb1bb2-27c9-409c-8a35-95d6d426462f service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:09:03.508980 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb1bb2-27c9-409c-8a35-95d6d426462f service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:09:03.513338 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13172730-ebf3-4e83-bcaf-f8fdebd9f291 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:09:03.513934 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13172730-ebf3-4e83-bcaf-f8fdebd9f291 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:09:03.514211 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13172730-ebf3-4e83-bcaf-f8fdebd9f291 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:09:03.514474 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13172730-ebf3-4e83-bcaf-f8fdebd9f291 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:09:03.525877 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13172730-ebf3-4e83-bcaf-f8fdebd9f291 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:09:03.527344 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13172730-ebf3-4e83-bcaf-f8fdebd9f291 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:09:03.538466 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1250/2494] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:09:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 08:09:04.725353 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6225e2-085e-4734-a136-fa034921e73b service nova] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:09:04.735745 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6225e2-085e-4734-a136-fa034921e73b service nova] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:09:04.740305 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ecca0f-c3bd-4d1d-b079-84248b1e2c6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:09:04.740932 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ecca0f-c3bd-4d1d-b079-84248b1e2c6e service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:09:04.741206 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ecca0f-c3bd-4d1d-b079-84248b1e2c6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:09:04.741468 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ecca0f-c3bd-4d1d-b079-84248b1e2c6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:09:04.752530 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ecca0f-c3bd-4d1d-b079-84248b1e2c6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:09:04.753947 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ecca0f-c3bd-4d1d-b079-84248b1e2c6e service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:09:04.765663 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1245/2495] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:09:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:09:08.630640 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13172730-ebf3-4e83-bcaf-f8fdebd9f291 service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:09:08.640117 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13172730-ebf3-4e83-bcaf-f8fdebd9f291 service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:09:08.644313 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839e54a-9196-49d8-b0a7-72be657a783c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:09:08.644956 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839e54a-9196-49d8-b0a7-72be657a783c service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:09:08.645259 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839e54a-9196-49d8-b0a7-72be657a783c service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:09:08.645528 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839e54a-9196-49d8-b0a7-72be657a783c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:09:08.653506 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c839e54a-9196-49d8-b0a7-72be657a783c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:09:08.654964 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c839e54a-9196-49d8-b0a7-72be657a783c service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:09:08.663173 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1251/2496] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:09:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:09:12.234463 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ecca0f-c3bd-4d1d-b079-84248b1e2c6e service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:09:12.242734 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ecca0f-c3bd-4d1d-b079-84248b1e2c6e service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:09:12.248554 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be876e-bd81-41b5-af47-3f4f1fec1bfa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6495385dccc4fa48c05b1feb957846c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:09:12.249163 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be876e-bd81-41b5-af47-3f4f1fec1bfa service nova] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:09:12.249496 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be876e-bd81-41b5-af47-3f4f1fec1bfa service nova] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:09:12.249765 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be876e-bd81-41b5-af47-3f4f1fec1bfa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:09:12.259287 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be876e-bd81-41b5-af47-3f4f1fec1bfa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:09:12.260852 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be876e-bd81-41b5-af47-3f4f1fec1bfa service nova] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:09:12.271474 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1246/2497] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:09:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:09:18.659742 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=18 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:09:22.267324 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=21 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:10:20.449118 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839e54a-9196-49d8-b0a7-72be657a783c service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:20.468213 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839e54a-9196-49d8-b0a7-72be657a783c service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:20.471882 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee96820b-693f-4d11-bb5a-d86685d83454 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:20.472705 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96820b-693f-4d11-bb5a-d86685d83454 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:20.473013 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96820b-693f-4d11-bb5a-d86685d83454 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:20.473280 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96820b-693f-4d11-bb5a-d86685d83454 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:20.483214 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee96820b-693f-4d11-bb5a-d86685d83454 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:20.484705 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee96820b-693f-4d11-bb5a-d86685d83454 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:20.495048 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1252/2498] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:10:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:10:23.751005 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be876e-bd81-41b5-af47-3f4f1fec1bfa service nova] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:23.773411 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be876e-bd81-41b5-af47-3f4f1fec1bfa service nova] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:23.780541 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab409f2-2f9c-4217-86c1-ffd36391ef14 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932d3076fb4b426d8c83d02eb60d8063', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:23.781139 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab409f2-2f9c-4217-86c1-ffd36391ef14 service placement] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:23.781450 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab409f2-2f9c-4217-86c1-ffd36391ef14 service placement] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:23.781719 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab409f2-2f9c-4217-86c1-ffd36391ef14 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:23.783760 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-0ab409f2-2f9c-4217-86c1-ffd36391ef14 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:10:23.807331 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee96820b-693f-4d11-bb5a-d86685d83454 service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:23.814744 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab409f2-2f9c-4217-86c1-ffd36391ef14 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:23.820535 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab409f2-2f9c-4217-86c1-ffd36391ef14 service placement] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:23.831315 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1247/2499] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:10:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:10:23.838288 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee96820b-693f-4d11-bb5a-d86685d83454 service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:23.843594 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49865646-1d1e-4372-899f-bc88a0776a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:23.844368 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49865646-1d1e-4372-899f-bc88a0776a84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:23.844666 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49865646-1d1e-4372-899f-bc88a0776a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:23.845037 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49865646-1d1e-4372-899f-bc88a0776a84 admin admin] PATH_INFO: `/v3/users/8972201328be4283af6873ec700dcf42` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:23.846180 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49865646-1d1e-4372-899f-bc88a0776a84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8972201328be4283af6873ec700dcf42)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:23.848341 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49865646-1d1e-4372-899f-bc88a0776a84 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:23.899702 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-49865646-1d1e-4372-899f-bc88a0776a84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8972201328be4283af6873ec700dcf42'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:23.906761 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-49865646-1d1e-4372-899f-bc88a0776a84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8972201328be4283af6873ec700dcf42'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:23.907499 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-49865646-1d1e-4372-899f-bc88a0776a84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8972201328be4283af6873ec700dcf42'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:23.921035 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-49865646-1d1e-4372-899f-bc88a0776a84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8972201328be4283af6873ec700dcf42'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:23.930920 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-49865646-1d1e-4372-899f-bc88a0776a84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8972201328be4283af6873ec700dcf42'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:23.937307 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1253/2500] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 08:10:23 2022] DELETE /identity/v3/users/8972201328be4283af6873ec700dcf42 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:10:24.137663 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab409f2-2f9c-4217-86c1-ffd36391ef14 service placement] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:24.149846 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab409f2-2f9c-4217-86c1-ffd36391ef14 service placement] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:24.221300 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f3117b-7fc3-4704-829f-bd92cfb9a3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:24.222289 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f3117b-7fc3-4704-829f-bd92cfb9a3e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:24.222794 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f3117b-7fc3-4704-829f-bd92cfb9a3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:24.223260 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f3117b-7fc3-4704-829f-bd92cfb9a3e1 admin admin] PATH_INFO: `/v3/projects/38d7aef5eecc465fae2a5158ec6aa6e2` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:24.225299 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f3117b-7fc3-4704-829f-bd92cfb9a3e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38d7aef5eecc465fae2a5158ec6aa6e2)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:24.227427 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f3117b-7fc3-4704-829f-bd92cfb9a3e1 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:24.287072 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-d4f3117b-7fc3-4704-829f-bd92cfb9a3e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38d7aef5eecc465fae2a5158ec6aa6e2', 'request_id': 'req-d4f3117b-7fc3-4704-829f-bd92cfb9a3e1'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:24.293966 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-d4f3117b-7fc3-4704-829f-bd92cfb9a3e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38d7aef5eecc465fae2a5158ec6aa6e2', 'request_id': 'req-d4f3117b-7fc3-4704-829f-bd92cfb9a3e1'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:24.300379 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-d4f3117b-7fc3-4704-829f-bd92cfb9a3e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38d7aef5eecc465fae2a5158ec6aa6e2', 'request_id': 'req-d4f3117b-7fc3-4704-829f-bd92cfb9a3e1'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:24.337095 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-d4f3117b-7fc3-4704-829f-bd92cfb9a3e1 admin admin] The token cache is being invalidate because project 38d7aef5eecc465fae2a5158ec6aa6e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 08:10:24.337352 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-d4f3117b-7fc3-4704-829f-bd92cfb9a3e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:24.338134 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-d4f3117b-7fc3-4704-829f-bd92cfb9a3e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:24.340208 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1248/2501] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 08:10:24 2022] DELETE /identity/v3/projects/38d7aef5eecc465fae2a5158ec6aa6e2 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:10:24.346207 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202f0ee-8c27-44ff-a6fc-73534466ca42 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:24.346554 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202f0ee-8c27-44ff-a6fc-73534466ca42 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:24.346909 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202f0ee-8c27-44ff-a6fc-73534466ca42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:24.369420 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-a202f0ee-8c27-44ff-a6fc-73534466ca42 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:10:24.370785 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-a202f0ee-8c27-44ff-a6fc-73534466ca42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:10:24.431846 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1254/2502] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:10:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 08:10:24.440092 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49481e-1e92-4c55-8460-3d2a864d85ab None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:24.440375 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49481e-1e92-4c55-8460-3d2a864d85ab None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:24.440633 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49481e-1e92-4c55-8460-3d2a864d85ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:24.461503 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-eb49481e-1e92-4c55-8460-3d2a864d85ab None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:10:24.462701 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-eb49481e-1e92-4c55-8460-3d2a864d85ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:10:24.471133 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1249/2503] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:10:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 08:10:24.475578 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202f0ee-8c27-44ff-a6fc-73534466ca42 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:24.485785 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202f0ee-8c27-44ff-a6fc-73534466ca42 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:24.491238 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45493baa-7f70-4e8f-bef0-11c5560a3ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:24.491853 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45493baa-7f70-4e8f-bef0-11c5560a3ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:24.492128 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45493baa-7f70-4e8f-bef0-11c5560a3ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:24.492401 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45493baa-7f70-4e8f-bef0-11c5560a3ab2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:24.493050 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45493baa-7f70-4e8f-bef0-11c5560a3ab2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 08:10:24.493352 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45493baa-7f70-4e8f-bef0-11c5560a3ab2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:24.495084 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45493baa-7f70-4e8f-bef0-11c5560a3ab2 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:24.506935 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1255/2504] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 08:10:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 08:10:24.511449 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49481e-1e92-4c55-8460-3d2a864d85ab None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:24.521065 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49481e-1e92-4c55-8460-3d2a864d85ab None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:24.526438 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c81327-e13e-46c8-84d0-65485a9378c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:24.527066 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c81327-e13e-46c8-84d0-65485a9378c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:24.527333 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c81327-e13e-46c8-84d0-65485a9378c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:24.527597 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c81327-e13e-46c8-84d0-65485a9378c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:24.528228 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c81327-e13e-46c8-84d0-65485a9378c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:24.530111 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c81327-e13e-46c8-84d0-65485a9378c8 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:24.561709 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1250/2505] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 08:10:24 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 08:10:24.567818 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45493baa-7f70-4e8f-bef0-11c5560a3ab2 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:24.579434 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45493baa-7f70-4e8f-bef0-11c5560a3ab2 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:24.599132 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:10:24.642088 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa6168-9623-4479-9e67-35d9981ad803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:24.642698 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa6168-9623-4479-9e67-35d9981ad803 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:24.643009 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa6168-9623-4479-9e67-35d9981ad803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:24.643275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa6168-9623-4479-9e67-35d9981ad803 admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:24.643902 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fa6168-9623-4479-9e67-35d9981ad803 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:24.645929 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fa6168-9623-4479-9e67-35d9981ad803 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:24.672450 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1256/2506] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 08:10:24 2022] POST /identity/v3/users => generated 291 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 08:10:24.677576 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c81327-e13e-46c8-84d0-65485a9378c8 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:24.693346 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c81327-e13e-46c8-84d0-65485a9378c8 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:24.698976 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd96a4-23c7-4e90-8352-feba942c229e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:24.699578 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd96a4-23c7-4e90-8352-feba942c229e admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:24.699851 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd96a4-23c7-4e90-8352-feba942c229e admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:24.700105 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd96a4-23c7-4e90-8352-feba942c229e admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:24.700774 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd96a4-23c7-4e90-8352-feba942c229e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:24.702375 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd96a4-23c7-4e90-8352-feba942c229e admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:24.713358 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1251/2507] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 08:10:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 08:10:24.717613 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa6168-9623-4479-9e67-35d9981ad803 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:24.728248 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa6168-9623-4479-9e67-35d9981ad803 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:24.734395 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5687808-7984-44b7-9ff1-59bee221a3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:24.735034 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5687808-7984-44b7-9ff1-59bee221a3ca admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:24.735312 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5687808-7984-44b7-9ff1-59bee221a3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:24.735584 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5687808-7984-44b7-9ff1-59bee221a3ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:24.736255 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5687808-7984-44b7-9ff1-59bee221a3ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:24.737896 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5687808-7984-44b7-9ff1-59bee221a3ca admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:24.749174 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1257/2508] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 08:10:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 08:10:24.755136 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd96a4-23c7-4e90-8352-feba942c229e admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:24.765553 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd96a4-23c7-4e90-8352-feba942c229e admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:24.770500 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0241261-f0c9-40f8-af35-cd1c418a5d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:24.771177 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0241261-f0c9-40f8-af35-cd1c418a5d0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:24.771447 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0241261-f0c9-40f8-af35-cd1c418a5d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:24.771733 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0241261-f0c9-40f8-af35-cd1c418a5d0f admin admin] PATH_INFO: `/v3/projects/11c22ba066c3470ea7c9f0f4a73dab78/users/a87073dcd52c4fbb8c707fb2b8ec88e7/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:24.773989 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:10:24.789478 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0241261-f0c9-40f8-af35-cd1c418a5d0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=11c22ba066c3470ea7c9f0f4a73dab78, user_id=a87073dcd52c4fbb8c707fb2b8ec88e7, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:24.793339 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0241261-f0c9-40f8-af35-cd1c418a5d0f admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:24.806302 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1252/2509] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 08:10:24 2022] PUT /identity/v3/projects/11c22ba066c3470ea7c9f0f4a73dab78/users/a87073dcd52c4fbb8c707fb2b8ec88e7/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:10:24.818143 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb3165-cc6a-4042-9b3e-3351f1ae2d52 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:24.818624 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb3165-cc6a-4042-9b3e-3351f1ae2d52 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:24.818624 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb3165-cc6a-4042-9b3e-3351f1ae2d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:24.847512 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-9adb3165-cc6a-4042-9b3e-3351f1ae2d52 None None] MFA Rules not processed for user `a87073dcd52c4fbb8c707fb2b8ec88e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:10:24.848910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-9adb3165-cc6a-4042-9b3e-3351f1ae2d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:10:24.917855 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1258/2510] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:10:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 08:10:24.922500 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0241261-f0c9-40f8-af35-cd1c418a5d0f admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:24.988345 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0241261-f0c9-40f8-af35-cd1c418a5d0f admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:24.993443 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af8226-b898-4726-bc77-e8b165ecfa45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:24.994052 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af8226-b898-4726-bc77-e8b165ecfa45 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:24.994322 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af8226-b898-4726-bc77-e8b165ecfa45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:24.994590 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af8226-b898-4726-bc77-e8b165ecfa45 admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:24.995234 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af8226-b898-4726-bc77-e8b165ecfa45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:24.997153 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af8226-b898-4726-bc77-e8b165ecfa45 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:25.028392 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1253/2511] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 08:10:24 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 08:10:25.033617 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb3165-cc6a-4042-9b3e-3351f1ae2d52 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:25.045540 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb3165-cc6a-4042-9b3e-3351f1ae2d52 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:25.106134 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ce893-3a9f-4419-8c02-5f7d0c7001fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:25.106854 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ce893-3a9f-4419-8c02-5f7d0c7001fe admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:25.107113 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ce893-3a9f-4419-8c02-5f7d0c7001fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:25.107381 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ce893-3a9f-4419-8c02-5f7d0c7001fe admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:25.107997 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290ce893-3a9f-4419-8c02-5f7d0c7001fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:25.109913 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290ce893-3a9f-4419-8c02-5f7d0c7001fe admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:25.133815 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1259/2512] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 08:10:25 2022] POST /identity/v3/users => generated 297 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 08:10:25.137730 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af8226-b898-4726-bc77-e8b165ecfa45 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:25.148469 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af8226-b898-4726-bc77-e8b165ecfa45 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:25.153148 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d25335-3d52-4eb2-987f-e1e8143bb5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:25.153751 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d25335-3d52-4eb2-987f-e1e8143bb5cd admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:25.154001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d25335-3d52-4eb2-987f-e1e8143bb5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:25.154270 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d25335-3d52-4eb2-987f-e1e8143bb5cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:25.154942 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d25335-3d52-4eb2-987f-e1e8143bb5cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:25.156545 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d25335-3d52-4eb2-987f-e1e8143bb5cd admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:25.167510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1254/2513] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 08:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 08:10:25.171713 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ce893-3a9f-4419-8c02-5f7d0c7001fe admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:25.183005 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ce893-3a9f-4419-8c02-5f7d0c7001fe admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:25.187971 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c416eb-083e-46f3-bded-26d1fb012d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:25.188626 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c416eb-083e-46f3-bded-26d1fb012d69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:25.188896 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c416eb-083e-46f3-bded-26d1fb012d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:25.189162 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c416eb-083e-46f3-bded-26d1fb012d69 admin admin] PATH_INFO: `/v3/projects/a2da1f2070bd439cbddf6c2bd467f790/users/228fa6dc7cf440ce8cd50cf1b35f52ac/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:25.207512 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c416eb-083e-46f3-bded-26d1fb012d69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2da1f2070bd439cbddf6c2bd467f790, user_id=228fa6dc7cf440ce8cd50cf1b35f52ac, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:25.210436 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c416eb-083e-46f3-bded-26d1fb012d69 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:25.212718 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:10:25.228218 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1260/2514] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 08:10:25 2022] PUT /identity/v3/projects/a2da1f2070bd439cbddf6c2bd467f790/users/228fa6dc7cf440ce8cd50cf1b35f52ac/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:10:25.232788 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d25335-3d52-4eb2-987f-e1e8143bb5cd admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:25.243470 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d25335-3d52-4eb2-987f-e1e8143bb5cd admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:25.305764 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e6fd4-51b1-40b7-8da2-079a4032bde7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:25.306362 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e6fd4-51b1-40b7-8da2-079a4032bde7 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:25.306633 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e6fd4-51b1-40b7-8da2-079a4032bde7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:25.306934 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e6fd4-51b1-40b7-8da2-079a4032bde7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:25.307566 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e6fd4-51b1-40b7-8da2-079a4032bde7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:25.309152 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e6fd4-51b1-40b7-8da2-079a4032bde7 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:25.319016 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1255/2515] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 08:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 08:10:25.323377 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c416eb-083e-46f3-bded-26d1fb012d69 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:25.333747 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c416eb-083e-46f3-bded-26d1fb012d69 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:25.338449 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a13b126-c21c-4369-81f2-5d35fbcc9691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:25.339137 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13b126-c21c-4369-81f2-5d35fbcc9691 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:25.339410 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13b126-c21c-4369-81f2-5d35fbcc9691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:25.339662 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13b126-c21c-4369-81f2-5d35fbcc9691 admin admin] PATH_INFO: `/v3/projects/a2da1f2070bd439cbddf6c2bd467f790/users/228fa6dc7cf440ce8cd50cf1b35f52ac/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:25.341534 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13b126-c21c-4369-81f2-5d35fbcc9691 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2da1f2070bd439cbddf6c2bd467f790, user_id=228fa6dc7cf440ce8cd50cf1b35f52ac, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:25.344180 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13b126-c21c-4369-81f2-5d35fbcc9691 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:25.354675 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1261/2516] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 08:10:25 2022] PUT /identity/v3/projects/a2da1f2070bd439cbddf6c2bd467f790/users/228fa6dc7cf440ce8cd50cf1b35f52ac/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:10:25.367079 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a68ad-a3dc-4d97-b152-4a8c8c60370d None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:25.367444 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a68ad-a3dc-4d97-b152-4a8c8c60370d None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:25.367800 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a68ad-a3dc-4d97-b152-4a8c8c60370d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:25.401984 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-f68a68ad-a3dc-4d97-b152-4a8c8c60370d None None] MFA Rules not processed for user `228fa6dc7cf440ce8cd50cf1b35f52ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:10:25.403226 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-f68a68ad-a3dc-4d97-b152-4a8c8c60370d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:10:25.447142 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:10:25.481595 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1256/2517] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:10:25 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 08:10:25.493808 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a13b126-c21c-4369-81f2-5d35fbcc9691 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:25.495771 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-4a13b126-c21c-4369-81f2-5d35fbcc9691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:10:25.576392 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a13b126-c21c-4369-81f2-5d35fbcc9691 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:25.580242 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830cfb2-1b72-460b-9857-4c5d364aa74a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:25.580853 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830cfb2-1b72-460b-9857-4c5d364aa74a service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:25.581130 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830cfb2-1b72-460b-9857-4c5d364aa74a service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:25.581398 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830cfb2-1b72-460b-9857-4c5d364aa74a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:25.598483 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3830cfb2-1b72-460b-9857-4c5d364aa74a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:25.600009 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3830cfb2-1b72-460b-9857-4c5d364aa74a service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:25.610883 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1262/2518] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:10:25 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:10:25.691896 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a68ad-a3dc-4d97-b152-4a8c8c60370d None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:25.702786 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a68ad-a3dc-4d97-b152-4a8c8c60370d None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:25.706598 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c401a8b-66d1-47fa-a311-c24126d8410b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:25.707239 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c401a8b-66d1-47fa-a311-c24126d8410b service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:25.707517 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c401a8b-66d1-47fa-a311-c24126d8410b service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:25.707786 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c401a8b-66d1-47fa-a311-c24126d8410b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:25.768405 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c401a8b-66d1-47fa-a311-c24126d8410b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:25.769855 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c401a8b-66d1-47fa-a311-c24126d8410b service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:25.779101 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1257/2519] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:10:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:10:35.606517 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:INSERT=2,keystone:SELECT=42 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:10:35.774837 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=38 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:10:35.835509 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830cfb2-1b72-460b-9857-4c5d364aa74a service cinder] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:35.908552 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830cfb2-1b72-460b-9857-4c5d364aa74a service cinder] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:35.913751 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609ae0f-2e9e-47bc-9255-ff206dbee079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:35.914384 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609ae0f-2e9e-47bc-9255-ff206dbee079 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:35.914686 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609ae0f-2e9e-47bc-9255-ff206dbee079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:35.914955 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609ae0f-2e9e-47bc-9255-ff206dbee079 admin admin] PATH_INFO: `/v3/users/a87073dcd52c4fbb8c707fb2b8ec88e7` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:35.916073 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5609ae0f-2e9e-47bc-9255-ff206dbee079 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a87073dcd52c4fbb8c707fb2b8ec88e7)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:35.917990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5609ae0f-2e9e-47bc-9255-ff206dbee079 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:35.974992 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5609ae0f-2e9e-47bc-9255-ff206dbee079 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a87073dcd52c4fbb8c707fb2b8ec88e7'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:35.982129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5609ae0f-2e9e-47bc-9255-ff206dbee079 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a87073dcd52c4fbb8c707fb2b8ec88e7'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:35.982822 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5609ae0f-2e9e-47bc-9255-ff206dbee079 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a87073dcd52c4fbb8c707fb2b8ec88e7'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:35.996485 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5609ae0f-2e9e-47bc-9255-ff206dbee079 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a87073dcd52c4fbb8c707fb2b8ec88e7'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:36.007412 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5609ae0f-2e9e-47bc-9255-ff206dbee079 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a87073dcd52c4fbb8c707fb2b8ec88e7'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:36.013959 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1263/2520] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 08:10:35 2022] DELETE /identity/v3/users/a87073dcd52c4fbb8c707fb2b8ec88e7 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:10:36.019971 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b40fc6-70cf-498a-a6bd-2886b4926a3f None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:36.020249 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b40fc6-70cf-498a-a6bd-2886b4926a3f None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:36.020523 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b40fc6-70cf-498a-a6bd-2886b4926a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:36.041200 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-a9b40fc6-70cf-498a-a6bd-2886b4926a3f None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:10:36.042385 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-a9b40fc6-70cf-498a-a6bd-2886b4926a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:10:36.106094 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1258/2521] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:10:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 08:10:36.119545 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609ae0f-2e9e-47bc-9255-ff206dbee079 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:36.121165 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-5609ae0f-2e9e-47bc-9255-ff206dbee079 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:10:36.208812 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609ae0f-2e9e-47bc-9255-ff206dbee079 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:36.212947 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45b454-2488-4cc6-b302-b5979d25093b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:36.213566 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45b454-2488-4cc6-b302-b5979d25093b service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:36.213840 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45b454-2488-4cc6-b302-b5979d25093b service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:36.214109 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45b454-2488-4cc6-b302-b5979d25093b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:36.223914 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa45b454-2488-4cc6-b302-b5979d25093b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:36.225347 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa45b454-2488-4cc6-b302-b5979d25093b service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:36.235894 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1264/2522] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:10:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 08:10:36.523927 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b40fc6-70cf-498a-a6bd-2886b4926a3f None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:36.525937 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-a9b40fc6-70cf-498a-a6bd-2886b4926a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:10:36.541983 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b40fc6-70cf-498a-a6bd-2886b4926a3f None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:36.547520 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af6a76-6a4f-466f-8fc0-d2e201e8b48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:36.548133 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af6a76-6a4f-466f-8fc0-d2e201e8b48f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:36.548409 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af6a76-6a4f-466f-8fc0-d2e201e8b48f admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:36.548677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af6a76-6a4f-466f-8fc0-d2e201e8b48f admin admin] PATH_INFO: `/v3/projects/11c22ba066c3470ea7c9f0f4a73dab78` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:36.550045 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99af6a76-6a4f-466f-8fc0-d2e201e8b48f admin admin] RBAC: Authorizing `identity:delete_project(project_id=11c22ba066c3470ea7c9f0f4a73dab78)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:36.551945 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99af6a76-6a4f-466f-8fc0-d2e201e8b48f admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:36.600834 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-99af6a76-6a4f-466f-8fc0-d2e201e8b48f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11c22ba066c3470ea7c9f0f4a73dab78', 'request_id': 'req-99af6a76-6a4f-466f-8fc0-d2e201e8b48f'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:36.607718 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-99af6a76-6a4f-466f-8fc0-d2e201e8b48f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11c22ba066c3470ea7c9f0f4a73dab78', 'request_id': 'req-99af6a76-6a4f-466f-8fc0-d2e201e8b48f'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:36.614204 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-99af6a76-6a4f-466f-8fc0-d2e201e8b48f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11c22ba066c3470ea7c9f0f4a73dab78', 'request_id': 'req-99af6a76-6a4f-466f-8fc0-d2e201e8b48f'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:36.633673 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-99af6a76-6a4f-466f-8fc0-d2e201e8b48f admin admin] The token cache is being invalidate because project 11c22ba066c3470ea7c9f0f4a73dab78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 08:10:36.634137 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-99af6a76-6a4f-466f-8fc0-d2e201e8b48f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:36.635253 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-99af6a76-6a4f-466f-8fc0-d2e201e8b48f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:36.637924 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1259/2523] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 08:10:36 2022] DELETE /identity/v3/projects/11c22ba066c3470ea7c9f0f4a73dab78 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 08:10:36.642089 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45b454-2488-4cc6-b302-b5979d25093b service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:36.649181 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:10:36.656851 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45b454-2488-4cc6-b302-b5979d25093b service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:36.718455 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8947dd8c-1aaf-476b-a5c4-88a4db5aa814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:36.719158 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8947dd8c-1aaf-476b-a5c4-88a4db5aa814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:36.719475 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8947dd8c-1aaf-476b-a5c4-88a4db5aa814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:36.719746 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8947dd8c-1aaf-476b-a5c4-88a4db5aa814 admin admin] PATH_INFO: `/v3/users/228fa6dc7cf440ce8cd50cf1b35f52ac` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:36.720913 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8947dd8c-1aaf-476b-a5c4-88a4db5aa814 admin admin] RBAC: Authorizing `identity:delete_user(user_id=228fa6dc7cf440ce8cd50cf1b35f52ac)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:36.722877 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8947dd8c-1aaf-476b-a5c4-88a4db5aa814 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:36.786141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-8947dd8c-1aaf-476b-a5c4-88a4db5aa814 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '228fa6dc7cf440ce8cd50cf1b35f52ac'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:36.793256 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-8947dd8c-1aaf-476b-a5c4-88a4db5aa814 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '228fa6dc7cf440ce8cd50cf1b35f52ac'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:36.793965 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-8947dd8c-1aaf-476b-a5c4-88a4db5aa814 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '228fa6dc7cf440ce8cd50cf1b35f52ac'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:36.806695 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-8947dd8c-1aaf-476b-a5c4-88a4db5aa814 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '228fa6dc7cf440ce8cd50cf1b35f52ac'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:36.815993 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-8947dd8c-1aaf-476b-a5c4-88a4db5aa814 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '228fa6dc7cf440ce8cd50cf1b35f52ac'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:36.822383 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1265/2524] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 08:10:36 2022] DELETE /identity/v3/users/228fa6dc7cf440ce8cd50cf1b35f52ac => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:10:37.113069 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af6a76-6a4f-466f-8fc0-d2e201e8b48f admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:37.114771 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-99af6a76-6a4f-466f-8fc0-d2e201e8b48f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:10:37.130934 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af6a76-6a4f-466f-8fc0-d2e201e8b48f admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:37.192186 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5064004-6905-4ac6-9820-c80dda22b91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:37.192827 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5064004-6905-4ac6-9820-c80dda22b91b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:37.193101 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5064004-6905-4ac6-9820-c80dda22b91b admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:37.193461 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5064004-6905-4ac6-9820-c80dda22b91b admin admin] PATH_INFO: `/v3/projects/a2da1f2070bd439cbddf6c2bd467f790` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:37.194552 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5064004-6905-4ac6-9820-c80dda22b91b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2da1f2070bd439cbddf6c2bd467f790)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:37.196510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5064004-6905-4ac6-9820-c80dda22b91b admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:37.232420 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:10:37.255658 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-f5064004-6905-4ac6-9820-c80dda22b91b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2da1f2070bd439cbddf6c2bd467f790', 'request_id': 'req-f5064004-6905-4ac6-9820-c80dda22b91b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:37.264759 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-f5064004-6905-4ac6-9820-c80dda22b91b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2da1f2070bd439cbddf6c2bd467f790', 'request_id': 'req-f5064004-6905-4ac6-9820-c80dda22b91b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:37.272205 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-f5064004-6905-4ac6-9820-c80dda22b91b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2da1f2070bd439cbddf6c2bd467f790', 'request_id': 'req-f5064004-6905-4ac6-9820-c80dda22b91b'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:37.287244 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-f5064004-6905-4ac6-9820-c80dda22b91b admin admin] The token cache is being invalidate because project a2da1f2070bd439cbddf6c2bd467f790 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 08:10:37.287607 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-f5064004-6905-4ac6-9820-c80dda22b91b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:37.288163 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-f5064004-6905-4ac6-9820-c80dda22b91b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:37.289661 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1260/2525] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 08:10:37 2022] DELETE /identity/v3/projects/a2da1f2070bd439cbddf6c2bd467f790 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:10:37.294597 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd70ce2b-d933-4ef1-a303-183b27c9d7c5 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:37.294898 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd70ce2b-d933-4ef1-a303-183b27c9d7c5 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:37.295163 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd70ce2b-d933-4ef1-a303-183b27c9d7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:37.315784 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-fd70ce2b-d933-4ef1-a303-183b27c9d7c5 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:10:37.317336 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-fd70ce2b-d933-4ef1-a303-183b27c9d7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:10:37.392229 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1266/2526] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:10:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 08:10:37.402214 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789be85-d8ed-4e0e-b841-0e11f5a3d3ba None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:37.402493 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789be85-d8ed-4e0e-b841-0e11f5a3d3ba None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:37.402804 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789be85-d8ed-4e0e-b841-0e11f5a3d3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:37.424072 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-8789be85-d8ed-4e0e-b841-0e11f5a3d3ba None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:10:37.425279 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-8789be85-d8ed-4e0e-b841-0e11f5a3d3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:10:37.434483 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1261/2527] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:10:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 08:10:37.438970 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd70ce2b-d933-4ef1-a303-183b27c9d7c5 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:37.455334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd70ce2b-d933-4ef1-a303-183b27c9d7c5 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:37.464250 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5032cab-be71-45b6-ba2a-c34407893c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:37.465098 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5032cab-be71-45b6-ba2a-c34407893c28 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:37.465443 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5032cab-be71-45b6-ba2a-c34407893c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:37.465773 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5032cab-be71-45b6-ba2a-c34407893c28 admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:37.466689 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5032cab-be71-45b6-ba2a-c34407893c28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 08:10:37.467052 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5032cab-be71-45b6-ba2a-c34407893c28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:37.469549 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5032cab-be71-45b6-ba2a-c34407893c28 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:37.482398 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1267/2528] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 08:10:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 08:10:37.488291 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8789be85-d8ed-4e0e-b841-0e11f5a3d3ba None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:37.498101 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8789be85-d8ed-4e0e-b841-0e11f5a3d3ba None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:37.503892 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039be1c4-5e2e-493a-86a4-8b847667017d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:37.504486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039be1c4-5e2e-493a-86a4-8b847667017d admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:37.504764 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039be1c4-5e2e-493a-86a4-8b847667017d admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:37.505025 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039be1c4-5e2e-493a-86a4-8b847667017d admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:37.505658 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039be1c4-5e2e-493a-86a4-8b847667017d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:37.507614 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039be1c4-5e2e-493a-86a4-8b847667017d admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:37.538172 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1262/2529] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 08:10:37 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 08:10:37.542404 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5032cab-be71-45b6-ba2a-c34407893c28 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:37.552613 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5032cab-be71-45b6-ba2a-c34407893c28 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:37.557273 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:10:37.614397 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e29461-730f-43ae-9b60-419ac7e857cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:37.615055 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e29461-730f-43ae-9b60-419ac7e857cc admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:37.615616 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e29461-730f-43ae-9b60-419ac7e857cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:37.615893 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e29461-730f-43ae-9b60-419ac7e857cc admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:37.616610 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e29461-730f-43ae-9b60-419ac7e857cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:37.618677 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e29461-730f-43ae-9b60-419ac7e857cc admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:37.642436 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1268/2530] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 08:10:37 2022] POST /identity/v3/users => generated 283 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 08:10:37.647010 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039be1c4-5e2e-493a-86a4-8b847667017d admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:37.658415 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039be1c4-5e2e-493a-86a4-8b847667017d admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:37.664445 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7893a4-b938-497c-9026-d3566c8a7270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:37.665049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7893a4-b938-497c-9026-d3566c8a7270 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:37.665323 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7893a4-b938-497c-9026-d3566c8a7270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:37.665584 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7893a4-b938-497c-9026-d3566c8a7270 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:37.666217 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7893a4-b938-497c-9026-d3566c8a7270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:37.667862 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7893a4-b938-497c-9026-d3566c8a7270 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:37.678228 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1263/2531] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 08:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 08:10:37.682529 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e29461-730f-43ae-9b60-419ac7e857cc admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:37.693143 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e29461-730f-43ae-9b60-419ac7e857cc admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:37.698699 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b9c55-3099-43da-96ec-0fa14e768d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:37.699321 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b9c55-3099-43da-96ec-0fa14e768d71 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:37.699592 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b9c55-3099-43da-96ec-0fa14e768d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:37.699860 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b9c55-3099-43da-96ec-0fa14e768d71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:37.700497 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b9c55-3099-43da-96ec-0fa14e768d71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:37.702102 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b9c55-3099-43da-96ec-0fa14e768d71 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:37.713109 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1269/2532] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 08:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 08:10:37.717517 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7893a4-b938-497c-9026-d3566c8a7270 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:37.728545 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7893a4-b938-497c-9026-d3566c8a7270 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:37.734699 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7264f-2dac-407f-8850-d9a21e3f9aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:37.735349 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7264f-2dac-407f-8850-d9a21e3f9aa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:37.735625 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7264f-2dac-407f-8850-d9a21e3f9aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:37.736032 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7264f-2dac-407f-8850-d9a21e3f9aa9 admin admin] PATH_INFO: `/v3/projects/43f254b1a7464e54b033427100ba5a3d/users/09d9d95f094d4aeb80059e9b82ee5fb7/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:37.752929 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a7264f-2dac-407f-8850-d9a21e3f9aa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43f254b1a7464e54b033427100ba5a3d, user_id=09d9d95f094d4aeb80059e9b82ee5fb7, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:37.755608 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a7264f-2dac-407f-8850-d9a21e3f9aa9 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:37.772190 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1264/2533] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 08:10:37 2022] PUT /identity/v3/projects/43f254b1a7464e54b033427100ba5a3d/users/09d9d95f094d4aeb80059e9b82ee5fb7/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 08:10:37.783389 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60678c1-3f13-4cd9-b07b-dea24e149af9 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:37.783679 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60678c1-3f13-4cd9-b07b-dea24e149af9 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:37.784007 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60678c1-3f13-4cd9-b07b-dea24e149af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:37.817580 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-c60678c1-3f13-4cd9-b07b-dea24e149af9 None None] MFA Rules not processed for user `09d9d95f094d4aeb80059e9b82ee5fb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:10:37.818870 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-c60678c1-3f13-4cd9-b07b-dea24e149af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:10:37.885313 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1270/2534] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:10:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 08:10:37.890082 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7264f-2dac-407f-8850-d9a21e3f9aa9 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:37.958241 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7264f-2dac-407f-8850-d9a21e3f9aa9 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:37.963232 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d44dd48-2e4d-4589-9046-6de4d8bd2b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:37.963836 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44dd48-2e4d-4589-9046-6de4d8bd2b79 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:37.964108 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44dd48-2e4d-4589-9046-6de4d8bd2b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:37.964380 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44dd48-2e4d-4589-9046-6de4d8bd2b79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:37.965017 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d44dd48-2e4d-4589-9046-6de4d8bd2b79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:37.966963 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d44dd48-2e4d-4589-9046-6de4d8bd2b79 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:37.998824 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1265/2535] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 08:10:37 2022] POST /identity/v3/projects => generated 363 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 08:10:38.004891 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60678c1-3f13-4cd9-b07b-dea24e149af9 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:38.018496 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60678c1-3f13-4cd9-b07b-dea24e149af9 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:38.078091 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a063d6-82f1-44e5-aa5e-0e608d791908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:38.078622 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a063d6-82f1-44e5-aa5e-0e608d791908 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:38.078924 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a063d6-82f1-44e5-aa5e-0e608d791908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:38.079223 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a063d6-82f1-44e5-aa5e-0e608d791908 admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:38.079845 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a063d6-82f1-44e5-aa5e-0e608d791908 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:38.081901 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a063d6-82f1-44e5-aa5e-0e608d791908 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:38.105148 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1271/2536] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 08:10:38 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 08:10:38.109838 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d44dd48-2e4d-4589-9046-6de4d8bd2b79 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:38.121587 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d44dd48-2e4d-4589-9046-6de4d8bd2b79 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:38.128317 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e595450-fa7c-42c8-93e7-0f0e6f537be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:38.128941 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e595450-fa7c-42c8-93e7-0f0e6f537be1 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:38.129211 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e595450-fa7c-42c8-93e7-0f0e6f537be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:38.129474 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e595450-fa7c-42c8-93e7-0f0e6f537be1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:38.130105 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e595450-fa7c-42c8-93e7-0f0e6f537be1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:38.131722 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e595450-fa7c-42c8-93e7-0f0e6f537be1 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:38.143230 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1266/2537] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 08:10:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 08:10:38.147621 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a063d6-82f1-44e5-aa5e-0e608d791908 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:38.157903 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a063d6-82f1-44e5-aa5e-0e608d791908 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:38.163327 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2370074f-1bdd-4055-ac64-f83c06a0e7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:38.164021 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370074f-1bdd-4055-ac64-f83c06a0e7bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:38.164286 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370074f-1bdd-4055-ac64-f83c06a0e7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:38.164561 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370074f-1bdd-4055-ac64-f83c06a0e7bc admin admin] PATH_INFO: `/v3/projects/3ab302a3421e4500b64dd4c763ca21df/users/a028367c15c046869e3aecaaf15d4012/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:38.166899 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:10:38.181553 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2370074f-1bdd-4055-ac64-f83c06a0e7bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ab302a3421e4500b64dd4c763ca21df, user_id=a028367c15c046869e3aecaaf15d4012, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:38.184248 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2370074f-1bdd-4055-ac64-f83c06a0e7bc admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:38.198613 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1272/2538] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 08:10:38 2022] PUT /identity/v3/projects/3ab302a3421e4500b64dd4c763ca21df/users/a028367c15c046869e3aecaaf15d4012/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:10:38.202568 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e595450-fa7c-42c8-93e7-0f0e6f537be1 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:38.212810 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e595450-fa7c-42c8-93e7-0f0e6f537be1 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:38.234581 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:10:38.275274 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee9718-7576-43d9-9d4f-693603000b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:38.275869 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee9718-7576-43d9-9d4f-693603000b30 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:38.276139 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee9718-7576-43d9-9d4f-693603000b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:38.276403 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee9718-7576-43d9-9d4f-693603000b30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:38.277026 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bee9718-7576-43d9-9d4f-693603000b30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:38.278634 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bee9718-7576-43d9-9d4f-693603000b30 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:38.288466 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1267/2539] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 08:10:38 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 08:10:38.292833 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2370074f-1bdd-4055-ac64-f83c06a0e7bc admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:38.303399 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2370074f-1bdd-4055-ac64-f83c06a0e7bc admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:38.308544 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa40c9f-b6c6-4b80-a66a-5e606216f600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:38.309187 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa40c9f-b6c6-4b80-a66a-5e606216f600 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:38.309459 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa40c9f-b6c6-4b80-a66a-5e606216f600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:38.309697 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa40c9f-b6c6-4b80-a66a-5e606216f600 admin admin] PATH_INFO: `/v3/projects/3ab302a3421e4500b64dd4c763ca21df/users/a028367c15c046869e3aecaaf15d4012/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:38.311882 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa40c9f-b6c6-4b80-a66a-5e606216f600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ab302a3421e4500b64dd4c763ca21df, user_id=a028367c15c046869e3aecaaf15d4012, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:38.314808 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa40c9f-b6c6-4b80-a66a-5e606216f600 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:38.325492 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1273/2540] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 08:10:38 2022] PUT /identity/v3/projects/3ab302a3421e4500b64dd4c763ca21df/users/a028367c15c046869e3aecaaf15d4012/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:10:38.337154 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28700e-c743-47f0-aeac-ca03b8178b9f None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:38.337468 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28700e-c743-47f0-aeac-ca03b8178b9f None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:38.337739 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28700e-c743-47f0-aeac-ca03b8178b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:38.368020 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-7b28700e-c743-47f0-aeac-ca03b8178b9f None None] MFA Rules not processed for user `a028367c15c046869e3aecaaf15d4012`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:10:38.369297 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-7b28700e-c743-47f0-aeac-ca03b8178b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:10:38.438600 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1268/2541] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:10:38 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 08:10:38.450986 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa40c9f-b6c6-4b80-a66a-5e606216f600 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:38.453195 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-3aa40c9f-b6c6-4b80-a66a-5e606216f600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:10:38.516671 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa40c9f-b6c6-4b80-a66a-5e606216f600 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:38.521235 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb1f59-4d4e-4f2c-a14e-7c1848fb588d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:38.521830 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb1f59-4d4e-4f2c-a14e-7c1848fb588d service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:38.522104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb1f59-4d4e-4f2c-a14e-7c1848fb588d service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:38.522371 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb1f59-4d4e-4f2c-a14e-7c1848fb588d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:38.532106 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edb1f59-4d4e-4f2c-a14e-7c1848fb588d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:38.534098 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edb1f59-4d4e-4f2c-a14e-7c1848fb588d service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:38.544069 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1274/2542] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:10:38 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:10:38.641155 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b28700e-c743-47f0-aeac-ca03b8178b9f None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:38.651973 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b28700e-c743-47f0-aeac-ca03b8178b9f None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:38.656475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f84c6-a368-42ac-aed7-5eb5af2a4dc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:38.657058 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f84c6-a368-42ac-aed7-5eb5af2a4dc1 service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:38.657326 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f84c6-a368-42ac-aed7-5eb5af2a4dc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:38.657617 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f84c6-a368-42ac-aed7-5eb5af2a4dc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:38.716611 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728f84c6-a368-42ac-aed7-5eb5af2a4dc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:38.718049 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728f84c6-a368-42ac-aed7-5eb5af2a4dc1 service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:38.727642 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1269/2543] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:10:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:10:48.540093 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=38,keystone:INSERT=2 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:10:48.723545 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=80 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:10:49.183378 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb1f59-4d4e-4f2c-a14e-7c1848fb588d service cinder] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:49.260644 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb1f59-4d4e-4f2c-a14e-7c1848fb588d service cinder] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:49.267023 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ad448-6bc2-4822-9c61-eac6d8c21f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:49.267602 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ad448-6bc2-4822-9c61-eac6d8c21f80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:49.267875 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ad448-6bc2-4822-9c61-eac6d8c21f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:49.268144 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ad448-6bc2-4822-9c61-eac6d8c21f80 admin admin] PATH_INFO: `/v3/users/09d9d95f094d4aeb80059e9b82ee5fb7` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:49.269437 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9ad448-6bc2-4822-9c61-eac6d8c21f80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09d9d95f094d4aeb80059e9b82ee5fb7)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:49.271419 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9ad448-6bc2-4822-9c61-eac6d8c21f80 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:49.327645 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-4a9ad448-6bc2-4822-9c61-eac6d8c21f80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09d9d95f094d4aeb80059e9b82ee5fb7'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:49.334480 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-4a9ad448-6bc2-4822-9c61-eac6d8c21f80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09d9d95f094d4aeb80059e9b82ee5fb7'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:49.335273 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-4a9ad448-6bc2-4822-9c61-eac6d8c21f80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09d9d95f094d4aeb80059e9b82ee5fb7'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:49.348176 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-4a9ad448-6bc2-4822-9c61-eac6d8c21f80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09d9d95f094d4aeb80059e9b82ee5fb7'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:49.357995 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-4a9ad448-6bc2-4822-9c61-eac6d8c21f80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09d9d95f094d4aeb80059e9b82ee5fb7'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:49.364000 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1275/2544] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 08:10:49 2022] DELETE /identity/v3/users/09d9d95f094d4aeb80059e9b82ee5fb7 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:10:49.368907 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62849de-944f-49f4-a36a-5155ee26eeea None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:49.369187 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62849de-944f-49f4-a36a-5155ee26eeea None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:49.369450 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62849de-944f-49f4-a36a-5155ee26eeea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:49.389317 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-d62849de-944f-49f4-a36a-5155ee26eeea None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:10:49.390513 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-d62849de-944f-49f4-a36a-5155ee26eeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:10:49.456283 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1270/2545] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:10:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 08:10:49.468829 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ad448-6bc2-4822-9c61-eac6d8c21f80 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:49.470416 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-4a9ad448-6bc2-4822-9c61-eac6d8c21f80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:10:49.529919 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ad448-6bc2-4822-9c61-eac6d8c21f80 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:49.534191 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66a5885-6c18-41ca-be1e-7e76380c087e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:49.534853 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a5885-6c18-41ca-be1e-7e76380c087e service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:49.535135 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a5885-6c18-41ca-be1e-7e76380c087e service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:49.535407 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a5885-6c18-41ca-be1e-7e76380c087e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:49.546537 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66a5885-6c18-41ca-be1e-7e76380c087e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:49.548064 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66a5885-6c18-41ca-be1e-7e76380c087e service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:49.558351 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1276/2546] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:10:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:10:49.852940 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62849de-944f-49f4-a36a-5155ee26eeea None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:49.854449 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-d62849de-944f-49f4-a36a-5155ee26eeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:10:49.869694 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62849de-944f-49f4-a36a-5155ee26eeea None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:49.875549 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5c530-ae4f-4944-807b-6437b7e4d4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:49.876286 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5c530-ae4f-4944-807b-6437b7e4d4e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:49.876462 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5c530-ae4f-4944-807b-6437b7e4d4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:49.876736 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5c530-ae4f-4944-807b-6437b7e4d4e5 admin admin] PATH_INFO: `/v3/projects/43f254b1a7464e54b033427100ba5a3d` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:49.877883 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b5c530-ae4f-4944-807b-6437b7e4d4e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43f254b1a7464e54b033427100ba5a3d)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:49.879831 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b5c530-ae4f-4944-807b-6437b7e4d4e5 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:49.929153 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a6b5c530-ae4f-4944-807b-6437b7e4d4e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43f254b1a7464e54b033427100ba5a3d', 'request_id': 'req-a6b5c530-ae4f-4944-807b-6437b7e4d4e5'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:49.935714 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a6b5c530-ae4f-4944-807b-6437b7e4d4e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43f254b1a7464e54b033427100ba5a3d', 'request_id': 'req-a6b5c530-ae4f-4944-807b-6437b7e4d4e5'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:49.941790 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a6b5c530-ae4f-4944-807b-6437b7e4d4e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43f254b1a7464e54b033427100ba5a3d', 'request_id': 'req-a6b5c530-ae4f-4944-807b-6437b7e4d4e5'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:49.956863 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a6b5c530-ae4f-4944-807b-6437b7e4d4e5 admin admin] The token cache is being invalidate because project 43f254b1a7464e54b033427100ba5a3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 08:10:49.957151 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a6b5c530-ae4f-4944-807b-6437b7e4d4e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:49.957763 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-a6b5c530-ae4f-4944-807b-6437b7e4d4e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:49.959297 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1271/2547] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 08:10:49 2022] DELETE /identity/v3/projects/43f254b1a7464e54b033427100ba5a3d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:10:49.963599 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66a5885-6c18-41ca-be1e-7e76380c087e service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:49.973586 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66a5885-6c18-41ca-be1e-7e76380c087e service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:49.993962 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:10:50.035084 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e212737-9941-42d3-b7d4-0da1bcb3a09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:50.035802 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e212737-9941-42d3-b7d4-0da1bcb3a09f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:50.036087 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e212737-9941-42d3-b7d4-0da1bcb3a09f admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:50.036370 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e212737-9941-42d3-b7d4-0da1bcb3a09f admin admin] PATH_INFO: `/v3/users/a028367c15c046869e3aecaaf15d4012` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:50.037593 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e212737-9941-42d3-b7d4-0da1bcb3a09f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a028367c15c046869e3aecaaf15d4012)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:50.039867 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e212737-9941-42d3-b7d4-0da1bcb3a09f admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:50.088149 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-6e212737-9941-42d3-b7d4-0da1bcb3a09f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a028367c15c046869e3aecaaf15d4012'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:50.094802 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-6e212737-9941-42d3-b7d4-0da1bcb3a09f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a028367c15c046869e3aecaaf15d4012'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:50.095449 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-6e212737-9941-42d3-b7d4-0da1bcb3a09f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a028367c15c046869e3aecaaf15d4012'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:50.107566 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-6e212737-9941-42d3-b7d4-0da1bcb3a09f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a028367c15c046869e3aecaaf15d4012'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:50.117571 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-6e212737-9941-42d3-b7d4-0da1bcb3a09f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a028367c15c046869e3aecaaf15d4012'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:50.123879 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1277/2548] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 08:10:49 2022] DELETE /identity/v3/users/a028367c15c046869e3aecaaf15d4012 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:10:50.405264 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5c530-ae4f-4944-807b-6437b7e4d4e5 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:50.406876 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-a6b5c530-ae4f-4944-807b-6437b7e4d4e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:10:50.422832 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5c530-ae4f-4944-807b-6437b7e4d4e5 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:50.484634 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d84a7-0aa2-4425-9541-dc74ef907c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:50.485260 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d84a7-0aa2-4425-9541-dc74ef907c87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:50.485535 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d84a7-0aa2-4425-9541-dc74ef907c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:50.485796 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d84a7-0aa2-4425-9541-dc74ef907c87 admin admin] PATH_INFO: `/v3/projects/3ab302a3421e4500b64dd4c763ca21df` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:50.486996 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122d84a7-0aa2-4425-9541-dc74ef907c87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ab302a3421e4500b64dd4c763ca21df)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:50.488960 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122d84a7-0aa2-4425-9541-dc74ef907c87 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:50.523249 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:10:50.540841 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-122d84a7-0aa2-4425-9541-dc74ef907c87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ab302a3421e4500b64dd4c763ca21df', 'request_id': 'req-122d84a7-0aa2-4425-9541-dc74ef907c87'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:50.547585 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-122d84a7-0aa2-4425-9541-dc74ef907c87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ab302a3421e4500b64dd4c763ca21df', 'request_id': 'req-122d84a7-0aa2-4425-9541-dc74ef907c87'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:50.553331 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-122d84a7-0aa2-4425-9541-dc74ef907c87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ab302a3421e4500b64dd4c763ca21df', 'request_id': 'req-122d84a7-0aa2-4425-9541-dc74ef907c87'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:50.567199 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-122d84a7-0aa2-4425-9541-dc74ef907c87 admin admin] The token cache is being invalidate because project 3ab302a3421e4500b64dd4c763ca21df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 08:10:50.567479 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-122d84a7-0aa2-4425-9541-dc74ef907c87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:50.568084 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-122d84a7-0aa2-4425-9541-dc74ef907c87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:50.569741 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1272/2549] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 08:10:50 2022] DELETE /identity/v3/projects/3ab302a3421e4500b64dd4c763ca21df => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:10:50.577372 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22a489-688d-4d4e-9c0a-75aaa0d0023a None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:50.578142 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22a489-688d-4d4e-9c0a-75aaa0d0023a None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:50.578142 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22a489-688d-4d4e-9c0a-75aaa0d0023a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:50.602216 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-6b22a489-688d-4d4e-9c0a-75aaa0d0023a None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:10:50.603486 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-6b22a489-688d-4d4e-9c0a-75aaa0d0023a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:10:50.676068 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1278/2550] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:10:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 08:10:50.751002 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b0742-8494-4a00-8f9c-752124ff3465 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:50.751282 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b0742-8494-4a00-8f9c-752124ff3465 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:50.751574 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b0742-8494-4a00-8f9c-752124ff3465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:50.776327 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-ee5b0742-8494-4a00-8f9c-752124ff3465 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:10:50.777563 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-ee5b0742-8494-4a00-8f9c-752124ff3465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:10:50.789109 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1273/2551] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:10:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 08:10:50.793787 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b22a489-688d-4d4e-9c0a-75aaa0d0023a None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:50.803857 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b22a489-688d-4d4e-9c0a-75aaa0d0023a None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:50.809889 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0c87d-bc6f-41b2-ba4f-1e6a25210c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:50.810513 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0c87d-bc6f-41b2-ba4f-1e6a25210c6c admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:50.810815 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0c87d-bc6f-41b2-ba4f-1e6a25210c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:50.811087 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0c87d-bc6f-41b2-ba4f-1e6a25210c6c admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:50.811730 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c0c87d-bc6f-41b2-ba4f-1e6a25210c6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 08:10:50.812030 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c0c87d-bc6f-41b2-ba4f-1e6a25210c6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:50.813632 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c0c87d-bc6f-41b2-ba4f-1e6a25210c6c admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:50.825902 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1279/2552] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 08:10:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 08:10:50.830531 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b0742-8494-4a00-8f9c-752124ff3465 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:50.840188 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b0742-8494-4a00-8f9c-752124ff3465 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:50.844976 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60635eb6-ae6a-4b32-86b8-0a4012166f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:50.845571 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60635eb6-ae6a-4b32-86b8-0a4012166f96 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:50.845826 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60635eb6-ae6a-4b32-86b8-0a4012166f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:50.846144 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60635eb6-ae6a-4b32-86b8-0a4012166f96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:50.846807 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60635eb6-ae6a-4b32-86b8-0a4012166f96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:50.848713 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60635eb6-ae6a-4b32-86b8-0a4012166f96 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:50.885104 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1274/2553] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 08:10:50 2022] POST /identity/v3/projects => generated 367 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 08:10:50.891087 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0c87d-bc6f-41b2-ba4f-1e6a25210c6c admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:50.907080 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0c87d-bc6f-41b2-ba4f-1e6a25210c6c admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:50.935968 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:10:50.971009 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4cdc5e-04b1-4cc4-aec5-e511a56e63f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:50.971624 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4cdc5e-04b1-4cc4-aec5-e511a56e63f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:50.971902 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4cdc5e-04b1-4cc4-aec5-e511a56e63f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:50.972198 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4cdc5e-04b1-4cc4-aec5-e511a56e63f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:50.972825 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4cdc5e-04b1-4cc4-aec5-e511a56e63f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:50.974793 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4cdc5e-04b1-4cc4-aec5-e511a56e63f7 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:50.999433 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1280/2554] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 08:10:50 2022] POST /identity/v3/users => generated 284 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 08:10:51.003954 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60635eb6-ae6a-4b32-86b8-0a4012166f96 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:51.014425 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60635eb6-ae6a-4b32-86b8-0a4012166f96 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:51.019133 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd52a05-aa39-4ce0-b366-a084d2f6d9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:51.019740 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd52a05-aa39-4ce0-b366-a084d2f6d9ec admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:51.020013 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd52a05-aa39-4ce0-b366-a084d2f6d9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:51.020303 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd52a05-aa39-4ce0-b366-a084d2f6d9ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:51.020923 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd52a05-aa39-4ce0-b366-a084d2f6d9ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:51.022515 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd52a05-aa39-4ce0-b366-a084d2f6d9ec admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:51.032687 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1275/2555] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 08:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 08:10:51.037529 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4cdc5e-04b1-4cc4-aec5-e511a56e63f7 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:51.053610 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4cdc5e-04b1-4cc4-aec5-e511a56e63f7 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:51.058881 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f7a8e-3da1-4049-88c6-0b246e7ddfe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:51.059488 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f7a8e-3da1-4049-88c6-0b246e7ddfe4 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:51.059772 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f7a8e-3da1-4049-88c6-0b246e7ddfe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:51.060022 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f7a8e-3da1-4049-88c6-0b246e7ddfe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:51.060659 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348f7a8e-3da1-4049-88c6-0b246e7ddfe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:51.062264 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348f7a8e-3da1-4049-88c6-0b246e7ddfe4 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:51.072647 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1281/2556] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 08:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 08:10:51.077039 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd52a05-aa39-4ce0-b366-a084d2f6d9ec admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:51.087204 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd52a05-aa39-4ce0-b366-a084d2f6d9ec admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:51.092820 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f95a53-cd12-43ac-84f8-f1f852c84060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:51.093465 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f95a53-cd12-43ac-84f8-f1f852c84060 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:51.093739 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f95a53-cd12-43ac-84f8-f1f852c84060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:51.094004 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f95a53-cd12-43ac-84f8-f1f852c84060 admin admin] PATH_INFO: `/v3/projects/7595c9bd922a4a728e0d95fd8748fdb9/users/2cb587efd2a746f29015e8c6c03b98b0/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:51.110383 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f95a53-cd12-43ac-84f8-f1f852c84060 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7595c9bd922a4a728e0d95fd8748fdb9, user_id=2cb587efd2a746f29015e8c6c03b98b0, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:51.113092 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f95a53-cd12-43ac-84f8-f1f852c84060 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:51.130308 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1276/2557] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 08:10:51 2022] PUT /identity/v3/projects/7595c9bd922a4a728e0d95fd8748fdb9/users/2cb587efd2a746f29015e8c6c03b98b0/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 08:10:51.142314 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb2397-5101-4dd2-8cf6-e082c726e125 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:51.142689 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb2397-5101-4dd2-8cf6-e082c726e125 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:51.143098 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb2397-5101-4dd2-8cf6-e082c726e125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:51.175481 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-86fb2397-5101-4dd2-8cf6-e082c726e125 None None] MFA Rules not processed for user `2cb587efd2a746f29015e8c6c03b98b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:10:51.176709 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-86fb2397-5101-4dd2-8cf6-e082c726e125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:10:51.250960 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1282/2558] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:10:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 08:10:51.255596 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f95a53-cd12-43ac-84f8-f1f852c84060 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:51.320392 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f95a53-cd12-43ac-84f8-f1f852c84060 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:51.326084 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297f981-6ede-475a-b5f0-fac88b7672de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:51.326720 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297f981-6ede-475a-b5f0-fac88b7672de admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:51.326989 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297f981-6ede-475a-b5f0-fac88b7672de admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:51.327249 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297f981-6ede-475a-b5f0-fac88b7672de admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:51.327858 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297f981-6ede-475a-b5f0-fac88b7672de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:51.329759 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297f981-6ede-475a-b5f0-fac88b7672de admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:51.364810 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1277/2559] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 08:10:51 2022] POST /identity/v3/projects => generated 365 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 08:10:51.369161 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb2397-5101-4dd2-8cf6-e082c726e125 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:51.380117 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb2397-5101-4dd2-8cf6-e082c726e125 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:51.444899 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd81b6-83e1-40c3-99af-c7b84a1984d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:51.445641 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd81b6-83e1-40c3-99af-c7b84a1984d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:51.445867 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd81b6-83e1-40c3-99af-c7b84a1984d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:51.446808 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd81b6-83e1-40c3-99af-c7b84a1984d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:51.446808 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fd81b6-83e1-40c3-99af-c7b84a1984d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:51.448715 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fd81b6-83e1-40c3-99af-c7b84a1984d7 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:51.471990 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1283/2560] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 08:10:51 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 08:10:51.476936 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297f981-6ede-475a-b5f0-fac88b7672de admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:51.488299 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297f981-6ede-475a-b5f0-fac88b7672de admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:51.493484 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb061a1b-0925-4954-9aa1-9a8a613641d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:51.494095 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb061a1b-0925-4954-9aa1-9a8a613641d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:51.494372 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb061a1b-0925-4954-9aa1-9a8a613641d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:51.494640 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb061a1b-0925-4954-9aa1-9a8a613641d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:51.495310 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb061a1b-0925-4954-9aa1-9a8a613641d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:51.496953 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb061a1b-0925-4954-9aa1-9a8a613641d6 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:51.508563 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1278/2561] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 08:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 08:10:51.513150 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd81b6-83e1-40c3-99af-c7b84a1984d7 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:51.523634 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd81b6-83e1-40c3-99af-c7b84a1984d7 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:51.529016 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6092be-4254-4c86-b8fe-5185dbb82b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:51.529666 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6092be-4254-4c86-b8fe-5185dbb82b7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:51.529948 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6092be-4254-4c86-b8fe-5185dbb82b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:51.530221 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6092be-4254-4c86-b8fe-5185dbb82b7e admin admin] PATH_INFO: `/v3/projects/1810a7bdb7a0465691594b2503fc4381/users/1a8a1347e7a5487abd3d9b91aae9bd88/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:51.548123 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6092be-4254-4c86-b8fe-5185dbb82b7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1810a7bdb7a0465691594b2503fc4381, user_id=1a8a1347e7a5487abd3d9b91aae9bd88, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:51.550891 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6092be-4254-4c86-b8fe-5185dbb82b7e admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:51.567556 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1284/2562] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 08:10:51 2022] PUT /identity/v3/projects/1810a7bdb7a0465691594b2503fc4381/users/1a8a1347e7a5487abd3d9b91aae9bd88/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:10:51.571935 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb061a1b-0925-4954-9aa1-9a8a613641d6 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:51.582252 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb061a1b-0925-4954-9aa1-9a8a613641d6 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:51.602975 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:10:51.643492 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c10f6a-3551-4cc4-b0e8-56328ef45040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:51.644048 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c10f6a-3551-4cc4-b0e8-56328ef45040 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:51.644308 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c10f6a-3551-4cc4-b0e8-56328ef45040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:51.644579 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c10f6a-3551-4cc4-b0e8-56328ef45040 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:51.645218 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c10f6a-3551-4cc4-b0e8-56328ef45040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:51.646849 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c10f6a-3551-4cc4-b0e8-56328ef45040 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:51.656696 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1279/2563] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 08:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 08:10:51.661339 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6092be-4254-4c86-b8fe-5185dbb82b7e admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:51.667303 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:10:51.673083 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6092be-4254-4c86-b8fe-5185dbb82b7e admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:51.679168 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0da536-55d3-48af-90d6-ef0d911dc38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:51.679833 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0da536-55d3-48af-90d6-ef0d911dc38e admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:51.680124 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0da536-55d3-48af-90d6-ef0d911dc38e admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:51.680378 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0da536-55d3-48af-90d6-ef0d911dc38e admin admin] PATH_INFO: `/v3/projects/1810a7bdb7a0465691594b2503fc4381/users/1a8a1347e7a5487abd3d9b91aae9bd88/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:51.682363 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0da536-55d3-48af-90d6-ef0d911dc38e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1810a7bdb7a0465691594b2503fc4381, user_id=1a8a1347e7a5487abd3d9b91aae9bd88, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:51.685069 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0da536-55d3-48af-90d6-ef0d911dc38e admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:51.695491 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1285/2564] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 08:10:51 2022] PUT /identity/v3/projects/1810a7bdb7a0465691594b2503fc4381/users/1a8a1347e7a5487abd3d9b91aae9bd88/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:10:51.707336 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2e52c-fa98-4646-91af-d28fc1aebfca None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:51.707699 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2e52c-fa98-4646-91af-d28fc1aebfca None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:51.708077 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2e52c-fa98-4646-91af-d28fc1aebfca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:51.742526 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-e7f2e52c-fa98-4646-91af-d28fc1aebfca None None] MFA Rules not processed for user `1a8a1347e7a5487abd3d9b91aae9bd88`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:10:51.743785 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-e7f2e52c-fa98-4646-91af-d28fc1aebfca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:10:51.816925 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1280/2565] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:10:51 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 08:10:51.834074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0da536-55d3-48af-90d6-ef0d911dc38e admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:51.836649 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-4f0da536-55d3-48af-90d6-ef0d911dc38e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:10:51.900515 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0da536-55d3-48af-90d6-ef0d911dc38e admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:51.904254 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbddcbd-0536-4746-abfc-c15e59a90aaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:51.904908 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbddcbd-0536-4746-abfc-c15e59a90aaf service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:51.905179 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbddcbd-0536-4746-abfc-c15e59a90aaf service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:51.905443 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbddcbd-0536-4746-abfc-c15e59a90aaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:51.915280 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbddcbd-0536-4746-abfc-c15e59a90aaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:51.916710 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbddcbd-0536-4746-abfc-c15e59a90aaf service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:51.926624 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1286/2566] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:10:51 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:10:52.029690 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2e52c-fa98-4646-91af-d28fc1aebfca None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:52.040481 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2e52c-fa98-4646-91af-d28fc1aebfca None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:52.045104 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5919e918-0a2d-4bf5-9ea2-9d2ab83f1167 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:52.045781 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919e918-0a2d-4bf5-9ea2-9d2ab83f1167 service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:52.046058 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919e918-0a2d-4bf5-9ea2-9d2ab83f1167 service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:52.046328 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919e918-0a2d-4bf5-9ea2-9d2ab83f1167 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:52.114924 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5919e918-0a2d-4bf5-9ea2-9d2ab83f1167 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:52.117194 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5919e918-0a2d-4bf5-9ea2-9d2ab83f1167 service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:52.126682 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1281/2567] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:10:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:10:57.034602 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbddcbd-0536-4746-abfc-c15e59a90aaf service cinder] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:57.098536 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbddcbd-0536-4746-abfc-c15e59a90aaf service cinder] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:57.103931 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac27bc-4a74-40c6-b6b7-c0d96980a59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:57.104557 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac27bc-4a74-40c6-b6b7-c0d96980a59d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:57.104832 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac27bc-4a74-40c6-b6b7-c0d96980a59d admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:57.105104 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac27bc-4a74-40c6-b6b7-c0d96980a59d admin admin] PATH_INFO: `/v3/users/2cb587efd2a746f29015e8c6c03b98b0` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:57.106829 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aac27bc-4a74-40c6-b6b7-c0d96980a59d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cb587efd2a746f29015e8c6c03b98b0)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:57.108755 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aac27bc-4a74-40c6-b6b7-c0d96980a59d admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:57.160151 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-3aac27bc-4a74-40c6-b6b7-c0d96980a59d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cb587efd2a746f29015e8c6c03b98b0'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:57.167008 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-3aac27bc-4a74-40c6-b6b7-c0d96980a59d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cb587efd2a746f29015e8c6c03b98b0'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:57.167681 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-3aac27bc-4a74-40c6-b6b7-c0d96980a59d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cb587efd2a746f29015e8c6c03b98b0'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:57.180087 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-3aac27bc-4a74-40c6-b6b7-c0d96980a59d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cb587efd2a746f29015e8c6c03b98b0'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:57.189247 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-3aac27bc-4a74-40c6-b6b7-c0d96980a59d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cb587efd2a746f29015e8c6c03b98b0'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:57.195469 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1287/2568] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 08:10:57 2022] DELETE /identity/v3/users/2cb587efd2a746f29015e8c6c03b98b0 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:10:57.200270 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2d58a-279a-4931-a544-e2e19c80b1a8 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:57.200763 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2d58a-279a-4931-a544-e2e19c80b1a8 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:57.200938 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2d58a-279a-4931-a544-e2e19c80b1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:57.224120 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-f2d2d58a-279a-4931-a544-e2e19c80b1a8 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:10:57.225318 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-f2d2d58a-279a-4931-a544-e2e19c80b1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:10:57.276222 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:10:57.298176 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1282/2569] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:10:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 08:10:57.308572 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac27bc-4a74-40c6-b6b7-c0d96980a59d admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:57.310516 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-3aac27bc-4a74-40c6-b6b7-c0d96980a59d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:10:57.326686 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:10:57.373248 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac27bc-4a74-40c6-b6b7-c0d96980a59d admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:57.377040 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a7f63-f59b-4bc7-8879-39b021ef441d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:57.377652 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a7f63-f59b-4bc7-8879-39b021ef441d service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:57.377929 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a7f63-f59b-4bc7-8879-39b021ef441d service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:57.378182 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a7f63-f59b-4bc7-8879-39b021ef441d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:57.387547 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7a7f63-f59b-4bc7-8879-39b021ef441d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:57.389194 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7a7f63-f59b-4bc7-8879-39b021ef441d service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:57.398937 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1288/2570] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:10:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:10:57.673203 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2d58a-279a-4931-a544-e2e19c80b1a8 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:57.674804 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-f2d2d58a-279a-4931-a544-e2e19c80b1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:10:57.687048 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2d58a-279a-4931-a544-e2e19c80b1a8 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:57.692777 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4b154-4760-486d-9343-1489d27314f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:57.693405 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4b154-4760-486d-9343-1489d27314f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:57.693677 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4b154-4760-486d-9343-1489d27314f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:57.693947 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4b154-4760-486d-9343-1489d27314f3 admin admin] PATH_INFO: `/v3/projects/7595c9bd922a4a728e0d95fd8748fdb9` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:57.695263 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4b154-4760-486d-9343-1489d27314f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7595c9bd922a4a728e0d95fd8748fdb9)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:57.697191 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4b154-4760-486d-9343-1489d27314f3 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:57.748600 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-fde4b154-4760-486d-9343-1489d27314f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7595c9bd922a4a728e0d95fd8748fdb9', 'request_id': 'req-fde4b154-4760-486d-9343-1489d27314f3'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:57.755377 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-fde4b154-4760-486d-9343-1489d27314f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7595c9bd922a4a728e0d95fd8748fdb9', 'request_id': 'req-fde4b154-4760-486d-9343-1489d27314f3'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:57.761465 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-fde4b154-4760-486d-9343-1489d27314f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7595c9bd922a4a728e0d95fd8748fdb9', 'request_id': 'req-fde4b154-4760-486d-9343-1489d27314f3'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:57.775758 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-fde4b154-4760-486d-9343-1489d27314f3 admin admin] The token cache is being invalidate because project 7595c9bd922a4a728e0d95fd8748fdb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 08:10:57.776026 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-fde4b154-4760-486d-9343-1489d27314f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:57.776628 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-fde4b154-4760-486d-9343-1489d27314f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:57.778154 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1283/2571] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 08:10:57 2022] DELETE /identity/v3/projects/7595c9bd922a4a728e0d95fd8748fdb9 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:10:57.782989 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a7f63-f59b-4bc7-8879-39b021ef441d service neutron] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:57.792879 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a7f63-f59b-4bc7-8879-39b021ef441d service neutron] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:57.857451 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9bf8b6-4ade-4d2e-9ac7-d24689843147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:57.858169 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9bf8b6-4ade-4d2e-9ac7-d24689843147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:57.858441 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9bf8b6-4ade-4d2e-9ac7-d24689843147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:57.858735 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9bf8b6-4ade-4d2e-9ac7-d24689843147 admin admin] PATH_INFO: `/v3/users/1a8a1347e7a5487abd3d9b91aae9bd88` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:57.860235 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9bf8b6-4ade-4d2e-9ac7-d24689843147 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a8a1347e7a5487abd3d9b91aae9bd88)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:57.862150 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9bf8b6-4ade-4d2e-9ac7-d24689843147 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:57.912484 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-3f9bf8b6-4ade-4d2e-9ac7-d24689843147 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a8a1347e7a5487abd3d9b91aae9bd88'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:57.920099 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-3f9bf8b6-4ade-4d2e-9ac7-d24689843147 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a8a1347e7a5487abd3d9b91aae9bd88'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:57.920684 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-3f9bf8b6-4ade-4d2e-9ac7-d24689843147 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a8a1347e7a5487abd3d9b91aae9bd88'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:57.934091 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-3f9bf8b6-4ade-4d2e-9ac7-d24689843147 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a8a1347e7a5487abd3d9b91aae9bd88'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:57.944590 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-3f9bf8b6-4ade-4d2e-9ac7-d24689843147 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a8a1347e7a5487abd3d9b91aae9bd88'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:57.954063 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1289/2572] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 08:10:57 2022] DELETE /identity/v3/users/1a8a1347e7a5487abd3d9b91aae9bd88 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:10:58.243819 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4b154-4760-486d-9343-1489d27314f3 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:58.245289 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-fde4b154-4760-486d-9343-1489d27314f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:10:58.262868 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4b154-4760-486d-9343-1489d27314f3 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:58.326110 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674f23a3-94f2-4645-b469-1e879a818396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:58.326799 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f23a3-94f2-4645-b469-1e879a818396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:58.327135 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f23a3-94f2-4645-b469-1e879a818396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:58.327342 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f23a3-94f2-4645-b469-1e879a818396 admin admin] PATH_INFO: `/v3/projects/1810a7bdb7a0465691594b2503fc4381` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:58.328578 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674f23a3-94f2-4645-b469-1e879a818396 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1810a7bdb7a0465691594b2503fc4381)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:58.330558 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674f23a3-94f2-4645-b469-1e879a818396 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:58.379904 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-674f23a3-94f2-4645-b469-1e879a818396 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1810a7bdb7a0465691594b2503fc4381', 'request_id': 'req-674f23a3-94f2-4645-b469-1e879a818396'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:58.386068 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-674f23a3-94f2-4645-b469-1e879a818396 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1810a7bdb7a0465691594b2503fc4381', 'request_id': 'req-674f23a3-94f2-4645-b469-1e879a818396'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:58.392231 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-674f23a3-94f2-4645-b469-1e879a818396 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1810a7bdb7a0465691594b2503fc4381', 'request_id': 'req-674f23a3-94f2-4645-b469-1e879a818396'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:58.401350 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:10:58.406744 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-674f23a3-94f2-4645-b469-1e879a818396 admin admin] The token cache is being invalidate because project 1810a7bdb7a0465691594b2503fc4381 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 08:10:58.407023 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-674f23a3-94f2-4645-b469-1e879a818396 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:58.407774 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-674f23a3-94f2-4645-b469-1e879a818396 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:10:58.409362 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1284/2573] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 08:10:58 2022] DELETE /identity/v3/projects/1810a7bdb7a0465691594b2503fc4381 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:10:58.414445 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93fedfb-280a-45be-aa28-ee8bc89b9305 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:58.414954 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93fedfb-280a-45be-aa28-ee8bc89b9305 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:58.415042 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93fedfb-280a-45be-aa28-ee8bc89b9305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:58.434871 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-e93fedfb-280a-45be-aa28-ee8bc89b9305 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:10:58.436121 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-e93fedfb-280a-45be-aa28-ee8bc89b9305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:10:58.465960 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:10:58.498922 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1290/2574] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:10:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 08:10:58.507862 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3f37c-8843-48da-8ffd-b2eb89ee8b8c None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:58.508177 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3f37c-8843-48da-8ffd-b2eb89ee8b8c None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:58.508468 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3f37c-8843-48da-8ffd-b2eb89ee8b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:58.537030 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-dfb3f37c-8843-48da-8ffd-b2eb89ee8b8c None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:10:58.538242 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-dfb3f37c-8843-48da-8ffd-b2eb89ee8b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:10:58.548181 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1285/2575] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:10:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 08:10:58.552732 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93fedfb-280a-45be-aa28-ee8bc89b9305 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:58.563402 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93fedfb-280a-45be-aa28-ee8bc89b9305 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:58.569014 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a71c5b-270e-4c99-94b1-66fe2d97ff30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:58.569613 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a71c5b-270e-4c99-94b1-66fe2d97ff30 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:58.569889 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a71c5b-270e-4c99-94b1-66fe2d97ff30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:58.570158 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a71c5b-270e-4c99-94b1-66fe2d97ff30 admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:58.570829 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a71c5b-270e-4c99-94b1-66fe2d97ff30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 08:10:58.571129 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a71c5b-270e-4c99-94b1-66fe2d97ff30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:58.572758 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a71c5b-270e-4c99-94b1-66fe2d97ff30 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:58.586403 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1291/2576] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 08:10:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 08:10:58.591051 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb3f37c-8843-48da-8ffd-b2eb89ee8b8c None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:58.600794 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb3f37c-8843-48da-8ffd-b2eb89ee8b8c None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:58.605843 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45675058-5117-42f8-b804-783e05826c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:58.606874 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45675058-5117-42f8-b804-783e05826c9a admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:58.607276 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45675058-5117-42f8-b804-783e05826c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:58.607749 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45675058-5117-42f8-b804-783e05826c9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:58.608395 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45675058-5117-42f8-b804-783e05826c9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:58.610287 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45675058-5117-42f8-b804-783e05826c9a admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:58.640636 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1286/2577] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 08:10:58 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 08:10:58.645179 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a71c5b-270e-4c99-94b1-66fe2d97ff30 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:58.655942 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a71c5b-270e-4c99-94b1-66fe2d97ff30 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:58.716586 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff504ca9-402b-4e52-ab59-4aae43465f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:58.717124 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff504ca9-402b-4e52-ab59-4aae43465f75 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:58.717398 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff504ca9-402b-4e52-ab59-4aae43465f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:58.717663 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff504ca9-402b-4e52-ab59-4aae43465f75 admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:58.718278 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff504ca9-402b-4e52-ab59-4aae43465f75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:58.720237 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff504ca9-402b-4e52-ab59-4aae43465f75 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:58.745724 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1292/2578] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 08:10:58 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 08:10:58.750609 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45675058-5117-42f8-b804-783e05826c9a admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:58.760915 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45675058-5117-42f8-b804-783e05826c9a admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:58.765981 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbaf08f-892c-4570-8912-13e8ba983a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:58.766575 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbaf08f-892c-4570-8912-13e8ba983a53 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:58.766869 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbaf08f-892c-4570-8912-13e8ba983a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:58.767134 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbaf08f-892c-4570-8912-13e8ba983a53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:58.767752 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbaf08f-892c-4570-8912-13e8ba983a53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:58.769369 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbaf08f-892c-4570-8912-13e8ba983a53 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:58.780326 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1287/2579] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 08:10:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 08:10:58.784383 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff504ca9-402b-4e52-ab59-4aae43465f75 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:58.794468 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff504ca9-402b-4e52-ab59-4aae43465f75 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:58.799900 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de745298-275f-422c-a0d3-1c0c83742549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:58.800490 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de745298-275f-422c-a0d3-1c0c83742549 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:58.800760 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de745298-275f-422c-a0d3-1c0c83742549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:58.801022 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de745298-275f-422c-a0d3-1c0c83742549 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:58.801653 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de745298-275f-422c-a0d3-1c0c83742549 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:58.803265 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de745298-275f-422c-a0d3-1c0c83742549 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:58.817106 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1293/2580] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 08:10:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 08:10:58.821169 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbaf08f-892c-4570-8912-13e8ba983a53 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:58.831276 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbaf08f-892c-4570-8912-13e8ba983a53 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:58.836336 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da772177-6a55-43dc-bf98-51593eaf00f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:58.836984 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da772177-6a55-43dc-bf98-51593eaf00f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:58.837254 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da772177-6a55-43dc-bf98-51593eaf00f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:58.837523 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da772177-6a55-43dc-bf98-51593eaf00f2 admin admin] PATH_INFO: `/v3/projects/4c059543fa5b4ab0a065f508f4a385b2/users/436a3c1df60143f591f4e90a960f993e/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:58.852758 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da772177-6a55-43dc-bf98-51593eaf00f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c059543fa5b4ab0a065f508f4a385b2, user_id=436a3c1df60143f591f4e90a960f993e, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:58.855421 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da772177-6a55-43dc-bf98-51593eaf00f2 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:58.871730 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1288/2581] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 08:10:58 2022] PUT /identity/v3/projects/4c059543fa5b4ab0a065f508f4a385b2/users/436a3c1df60143f591f4e90a960f993e/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:10:58.881786 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2f48f-9236-4fcf-9ccb-a627a2afba6f None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:58.882066 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2f48f-9236-4fcf-9ccb-a627a2afba6f None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:58.882333 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2f48f-9236-4fcf-9ccb-a627a2afba6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:58.912326 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-ffb2f48f-9236-4fcf-9ccb-a627a2afba6f None None] MFA Rules not processed for user `436a3c1df60143f591f4e90a960f993e`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:10:58.913528 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-ffb2f48f-9236-4fcf-9ccb-a627a2afba6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:10:58.983112 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1294/2582] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:10:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 08:10:58.994541 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da772177-6a55-43dc-bf98-51593eaf00f2 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:10:58.996523 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-da772177-6a55-43dc-bf98-51593eaf00f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:10:59.054726 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da772177-6a55-43dc-bf98-51593eaf00f2 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:10:59.058582 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc646969-e401-418f-a185-32e72b3f6249 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:10:59.059214 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc646969-e401-418f-a185-32e72b3f6249 service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:10:59.059485 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc646969-e401-418f-a185-32e72b3f6249 service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:10:59.059753 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc646969-e401-418f-a185-32e72b3f6249 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:10:59.068473 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc646969-e401-418f-a185-32e72b3f6249 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:10:59.069898 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc646969-e401-418f-a185-32e72b3f6249 service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:10:59.078537 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1289/2583] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:10:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:11:08.979718 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:11:09.074708 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=54,keystone:INSERT=2 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:11:20.256773 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb2f48f-9236-4fcf-9ccb-a627a2afba6f None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:20.332059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb2f48f-9236-4fcf-9ccb-a627a2afba6f None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:20.336958 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5034911-7b87-42d9-a80e-08234ad1922e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:20.337580 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5034911-7b87-42d9-a80e-08234ad1922e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:20.337852 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5034911-7b87-42d9-a80e-08234ad1922e admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:20.338118 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5034911-7b87-42d9-a80e-08234ad1922e admin admin] PATH_INFO: `/v3/users/436a3c1df60143f591f4e90a960f993e` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:20.339458 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5034911-7b87-42d9-a80e-08234ad1922e admin admin] RBAC: Authorizing `identity:delete_user(user_id=436a3c1df60143f591f4e90a960f993e)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:20.341575 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5034911-7b87-42d9-a80e-08234ad1922e admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:20.391533 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-a5034911-7b87-42d9-a80e-08234ad1922e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '436a3c1df60143f591f4e90a960f993e'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:20.398836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-a5034911-7b87-42d9-a80e-08234ad1922e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '436a3c1df60143f591f4e90a960f993e'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:20.399576 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-a5034911-7b87-42d9-a80e-08234ad1922e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '436a3c1df60143f591f4e90a960f993e'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:20.413177 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-a5034911-7b87-42d9-a80e-08234ad1922e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '436a3c1df60143f591f4e90a960f993e'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:20.421730 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-a5034911-7b87-42d9-a80e-08234ad1922e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '436a3c1df60143f591f4e90a960f993e'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:20.427682 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1295/2584] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 08:11:20 2022] DELETE /identity/v3/users/436a3c1df60143f591f4e90a960f993e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:11:20.432294 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22005839-31e5-4248-bb6c-0ba37152faca None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:20.433045 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22005839-31e5-4248-bb6c-0ba37152faca None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:20.433045 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22005839-31e5-4248-bb6c-0ba37152faca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:20.452444 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-22005839-31e5-4248-bb6c-0ba37152faca None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:11:20.453643 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-22005839-31e5-4248-bb6c-0ba37152faca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:11:20.524919 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1290/2585] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:11:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 08:11:20.535963 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5034911-7b87-42d9-a80e-08234ad1922e admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:20.537457 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-a5034911-7b87-42d9-a80e-08234ad1922e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:11:20.597636 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5034911-7b87-42d9-a80e-08234ad1922e admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:20.602275 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f25d0-d6e7-4540-8348-d2647059ec97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:20.602915 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f25d0-d6e7-4540-8348-d2647059ec97 service neutron] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:20.603191 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f25d0-d6e7-4540-8348-d2647059ec97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:20.603459 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f25d0-d6e7-4540-8348-d2647059ec97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:20.613864 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f25d0-d6e7-4540-8348-d2647059ec97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:20.615336 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f25d0-d6e7-4540-8348-d2647059ec97 service neutron] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:20.625336 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1296/2586] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:11:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:11:20.931714 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22005839-31e5-4248-bb6c-0ba37152faca None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:20.933162 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-22005839-31e5-4248-bb6c-0ba37152faca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:11:20.945637 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22005839-31e5-4248-bb6c-0ba37152faca None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:20.951116 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45216ce-6d82-4e5e-a468-0e07d52b2d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:20.951758 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45216ce-6d82-4e5e-a468-0e07d52b2d06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:20.952033 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45216ce-6d82-4e5e-a468-0e07d52b2d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:20.952298 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45216ce-6d82-4e5e-a468-0e07d52b2d06 admin admin] PATH_INFO: `/v3/projects/4c059543fa5b4ab0a065f508f4a385b2` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:20.953646 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45216ce-6d82-4e5e-a468-0e07d52b2d06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c059543fa5b4ab0a065f508f4a385b2)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:20.955498 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45216ce-6d82-4e5e-a468-0e07d52b2d06 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:21.007007 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c45216ce-6d82-4e5e-a468-0e07d52b2d06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c059543fa5b4ab0a065f508f4a385b2', 'request_id': 'req-c45216ce-6d82-4e5e-a468-0e07d52b2d06'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:21.013261 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c45216ce-6d82-4e5e-a468-0e07d52b2d06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c059543fa5b4ab0a065f508f4a385b2', 'request_id': 'req-c45216ce-6d82-4e5e-a468-0e07d52b2d06'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:21.018736 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c45216ce-6d82-4e5e-a468-0e07d52b2d06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c059543fa5b4ab0a065f508f4a385b2', 'request_id': 'req-c45216ce-6d82-4e5e-a468-0e07d52b2d06'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:21.033602 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c45216ce-6d82-4e5e-a468-0e07d52b2d06 admin admin] The token cache is being invalidate because project 4c059543fa5b4ab0a065f508f4a385b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 08:11:21.033924 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c45216ce-6d82-4e5e-a468-0e07d52b2d06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:21.034614 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c45216ce-6d82-4e5e-a468-0e07d52b2d06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:21.036243 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1291/2587] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 08:11:20 2022] DELETE /identity/v3/projects/4c059543fa5b4ab0a065f508f4a385b2 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:11:21.041311 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8632e7-4d33-445a-bc66-a8c036a3e633 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:21.041595 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8632e7-4d33-445a-bc66-a8c036a3e633 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:21.041859 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8632e7-4d33-445a-bc66-a8c036a3e633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:21.062105 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-4c8632e7-4d33-445a-bc66-a8c036a3e633 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:11:21.063349 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-4c8632e7-4d33-445a-bc66-a8c036a3e633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:11:21.082176 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:11:21.129675 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1297/2588] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 08:11:21.139825 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d071c0-b1fa-44c1-842b-6f3769a6b984 None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:21.140373 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d071c0-b1fa-44c1-842b-6f3769a6b984 None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:21.140771 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d071c0-b1fa-44c1-842b-6f3769a6b984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:21.166455 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-a1d071c0-b1fa-44c1-842b-6f3769a6b984 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:11:21.167738 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-a1d071c0-b1fa-44c1-842b-6f3769a6b984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:11:21.177830 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1292/2589] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 08:11:21.183074 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8632e7-4d33-445a-bc66-a8c036a3e633 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:21.194243 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8632e7-4d33-445a-bc66-a8c036a3e633 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:21.200419 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af90141-1ae0-4a8c-ae07-ac0f0618588a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:21.201023 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af90141-1ae0-4a8c-ae07-ac0f0618588a admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:21.201270 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af90141-1ae0-4a8c-ae07-ac0f0618588a admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:21.201537 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af90141-1ae0-4a8c-ae07-ac0f0618588a admin admin] PATH_INFO: `/v3/domains` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:21.202183 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af90141-1ae0-4a8c-ae07-ac0f0618588a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 08:11:21.202487 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af90141-1ae0-4a8c-ae07-ac0f0618588a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:21.204147 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af90141-1ae0-4a8c-ae07-ac0f0618588a admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:21.217142 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1298/2590] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 08:11:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 08:11:21.221873 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d071c0-b1fa-44c1-842b-6f3769a6b984 None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:21.232084 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d071c0-b1fa-44c1-842b-6f3769a6b984 None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:21.238982 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e72d21-a280-4dc2-b7e6-0f47c4910d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:21.240341 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e72d21-a280-4dc2-b7e6-0f47c4910d2e admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:21.240341 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e72d21-a280-4dc2-b7e6-0f47c4910d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:21.240675 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e72d21-a280-4dc2-b7e6-0f47c4910d2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:21.241338 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e72d21-a280-4dc2-b7e6-0f47c4910d2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:21.243348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e72d21-a280-4dc2-b7e6-0f47c4910d2e admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:21.274393 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1293/2591] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 08:11:21 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 08:11:21.278647 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af90141-1ae0-4a8c-ae07-ac0f0618588a admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:21.289754 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af90141-1ae0-4a8c-ae07-ac0f0618588a admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:21.350014 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f70ca7-1f84-4617-b7f9-84eb5db68dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:21.350620 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f70ca7-1f84-4617-b7f9-84eb5db68dbf admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:21.350929 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f70ca7-1f84-4617-b7f9-84eb5db68dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:21.351202 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f70ca7-1f84-4617-b7f9-84eb5db68dbf admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:21.351841 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f70ca7-1f84-4617-b7f9-84eb5db68dbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:21.353763 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f70ca7-1f84-4617-b7f9-84eb5db68dbf admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:21.379106 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1299/2592] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 08:11:21 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 08:11:21.383805 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e72d21-a280-4dc2-b7e6-0f47c4910d2e admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:21.396107 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e72d21-a280-4dc2-b7e6-0f47c4910d2e admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:21.401733 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad52db-e12e-41d1-8e13-7379be64b3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:21.402369 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad52db-e12e-41d1-8e13-7379be64b3db admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:21.402621 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad52db-e12e-41d1-8e13-7379be64b3db admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:21.402954 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad52db-e12e-41d1-8e13-7379be64b3db admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:21.403635 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ad52db-e12e-41d1-8e13-7379be64b3db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:21.405283 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ad52db-e12e-41d1-8e13-7379be64b3db admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:21.415920 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1294/2593] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 08:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 08:11:21.421721 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f70ca7-1f84-4617-b7f9-84eb5db68dbf admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:21.432694 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f70ca7-1f84-4617-b7f9-84eb5db68dbf admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:21.438986 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8cdbd1-1cf6-4a3e-909f-803bf52e881c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:21.439698 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cdbd1-1cf6-4a3e-909f-803bf52e881c admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:21.439973 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cdbd1-1cf6-4a3e-909f-803bf52e881c admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:21.440243 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cdbd1-1cf6-4a3e-909f-803bf52e881c admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:21.440882 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8cdbd1-1cf6-4a3e-909f-803bf52e881c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:21.442490 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8cdbd1-1cf6-4a3e-909f-803bf52e881c admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:21.455050 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1300/2594] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 08:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 08:11:21.461537 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad52db-e12e-41d1-8e13-7379be64b3db admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:21.476577 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad52db-e12e-41d1-8e13-7379be64b3db admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:21.483825 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df940d-9919-4587-8e14-f95bcea54260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:21.484475 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df940d-9919-4587-8e14-f95bcea54260 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:21.484751 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df940d-9919-4587-8e14-f95bcea54260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:21.485026 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df940d-9919-4587-8e14-f95bcea54260 admin admin] PATH_INFO: `/v3/projects/fabd0ccc1f254228aae5e15bf373409f/users/c461c67cd0b64d5fa14cf6e3bc6203d4/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:21.501949 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df940d-9919-4587-8e14-f95bcea54260 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fabd0ccc1f254228aae5e15bf373409f, user_id=c461c67cd0b64d5fa14cf6e3bc6203d4, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:21.504644 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df940d-9919-4587-8e14-f95bcea54260 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:21.520329 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1295/2595] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 08:11:21 2022] PUT /identity/v3/projects/fabd0ccc1f254228aae5e15bf373409f/users/c461c67cd0b64d5fa14cf6e3bc6203d4/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:11:21.530576 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9226c46b-bf4c-49ec-ba5d-0ea8e0f200f5 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:21.530795 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9226c46b-bf4c-49ec-ba5d-0ea8e0f200f5 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:21.531095 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9226c46b-bf4c-49ec-ba5d-0ea8e0f200f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:21.565289 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-9226c46b-bf4c-49ec-ba5d-0ea8e0f200f5 None None] MFA Rules not processed for user `c461c67cd0b64d5fa14cf6e3bc6203d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:11:21.567163 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-9226c46b-bf4c-49ec-ba5d-0ea8e0f200f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:11:21.646643 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1301/2596] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:11:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 08:11:21.651487 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df940d-9919-4587-8e14-f95bcea54260 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:21.674017 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:11:21.719302 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df940d-9919-4587-8e14-f95bcea54260 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:21.724461 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871f6b8-4cf7-4d6d-bd8c-cf970556e2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:21.725069 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871f6b8-4cf7-4d6d-bd8c-cf970556e2d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:21.725347 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871f6b8-4cf7-4d6d-bd8c-cf970556e2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:21.725622 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871f6b8-4cf7-4d6d-bd8c-cf970556e2d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:21.726250 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871f6b8-4cf7-4d6d-bd8c-cf970556e2d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:21.728190 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871f6b8-4cf7-4d6d-bd8c-cf970556e2d7 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:21.757373 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1296/2597] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 08:11:21 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 08:11:21.761621 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9226c46b-bf4c-49ec-ba5d-0ea8e0f200f5 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:21.772377 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9226c46b-bf4c-49ec-ba5d-0ea8e0f200f5 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:21.785068 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:11:21.838634 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e545232-2e6f-4dd7-a9ba-2310b8a1d0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:21.839272 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e545232-2e6f-4dd7-a9ba-2310b8a1d0a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:21.839553 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e545232-2e6f-4dd7-a9ba-2310b8a1d0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:21.839821 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e545232-2e6f-4dd7-a9ba-2310b8a1d0a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:21.840578 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e545232-2e6f-4dd7-a9ba-2310b8a1d0a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:21.842593 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e545232-2e6f-4dd7-a9ba-2310b8a1d0a4 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:21.872121 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1302/2598] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 08:11:21 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 08:11:21.876428 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871f6b8-4cf7-4d6d-bd8c-cf970556e2d7 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:21.888144 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871f6b8-4cf7-4d6d-bd8c-cf970556e2d7 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:21.893200 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857dde7-0d2b-470b-b538-d513e448abeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:21.893805 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857dde7-0d2b-470b-b538-d513e448abeb admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:21.894081 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857dde7-0d2b-470b-b538-d513e448abeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:21.894348 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857dde7-0d2b-470b-b538-d513e448abeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:21.895027 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6857dde7-0d2b-470b-b538-d513e448abeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:21.896645 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6857dde7-0d2b-470b-b538-d513e448abeb admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:21.908006 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1297/2599] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 08:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 08:11:21.913439 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e545232-2e6f-4dd7-a9ba-2310b8a1d0a4 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:21.923985 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e545232-2e6f-4dd7-a9ba-2310b8a1d0a4 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:21.928940 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12edcd14-d4fe-4f54-8c31-db9ccd1d6a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:21.929533 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12edcd14-d4fe-4f54-8c31-db9ccd1d6a57 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:21.929815 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12edcd14-d4fe-4f54-8c31-db9ccd1d6a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:21.930086 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12edcd14-d4fe-4f54-8c31-db9ccd1d6a57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:21.930760 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12edcd14-d4fe-4f54-8c31-db9ccd1d6a57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:21.932344 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12edcd14-d4fe-4f54-8c31-db9ccd1d6a57 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:21.942677 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1303/2600] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 08:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 08:11:21.946900 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857dde7-0d2b-470b-b538-d513e448abeb admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:21.957604 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857dde7-0d2b-470b-b538-d513e448abeb admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:21.962937 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf22d2-99bb-42ab-85ed-42bb9a195c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:21.963586 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf22d2-99bb-42ab-85ed-42bb9a195c79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:21.963861 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf22d2-99bb-42ab-85ed-42bb9a195c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:21.964119 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf22d2-99bb-42ab-85ed-42bb9a195c79 admin admin] PATH_INFO: `/v3/projects/e3110822fe994d0594692b628df8ec87/users/039badaf49a74b4c970196f7b191c1b8/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:21.979352 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bf22d2-99bb-42ab-85ed-42bb9a195c79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3110822fe994d0594692b628df8ec87, user_id=039badaf49a74b4c970196f7b191c1b8, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:21.983865 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bf22d2-99bb-42ab-85ed-42bb9a195c79 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:21.999597 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1298/2601] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 08:11:21 2022] PUT /identity/v3/projects/e3110822fe994d0594692b628df8ec87/users/039badaf49a74b4c970196f7b191c1b8/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 08:11:22.011159 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd991c-7c3b-42fc-bc94-a909abad404a None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:22.011442 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd991c-7c3b-42fc-bc94-a909abad404a None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:22.011710 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd991c-7c3b-42fc-bc94-a909abad404a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:22.041299 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-5bcd991c-7c3b-42fc-bc94-a909abad404a None None] MFA Rules not processed for user `039badaf49a74b4c970196f7b191c1b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:11:22.042513 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-5bcd991c-7c3b-42fc-bc94-a909abad404a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:11:22.110729 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1304/2602] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:11:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 08:11:22.115585 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf22d2-99bb-42ab-85ed-42bb9a195c79 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:22.190525 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf22d2-99bb-42ab-85ed-42bb9a195c79 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:22.195778 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f3821-22a3-4015-a533-bb26fbc32a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:22.196381 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f3821-22a3-4015-a533-bb26fbc32a0b admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:22.196652 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f3821-22a3-4015-a533-bb26fbc32a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:22.196911 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f3821-22a3-4015-a533-bb26fbc32a0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:22.197525 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3f3821-22a3-4015-a533-bb26fbc32a0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:22.199596 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3f3821-22a3-4015-a533-bb26fbc32a0b admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:22.230525 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1299/2603] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 08:11:22 2022] POST /identity/v3/projects => generated 377 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 08:11:22.235548 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd991c-7c3b-42fc-bc94-a909abad404a None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:22.246645 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd991c-7c3b-42fc-bc94-a909abad404a None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:22.318683 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8fa7f-0ac0-4a89-98f5-e266b819b80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:22.319340 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8fa7f-0ac0-4a89-98f5-e266b819b80c admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:22.319455 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8fa7f-0ac0-4a89-98f5-e266b819b80c admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:22.319879 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8fa7f-0ac0-4a89-98f5-e266b819b80c admin admin] PATH_INFO: `/v3/users` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:22.320507 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa8fa7f-0ac0-4a89-98f5-e266b819b80c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:22.322425 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa8fa7f-0ac0-4a89-98f5-e266b819b80c admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:22.345749 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1305/2604] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 08:11:22 2022] POST /identity/v3/users => generated 295 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 08:11:22.349995 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f3821-22a3-4015-a533-bb26fbc32a0b admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:22.361233 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f3821-22a3-4015-a533-bb26fbc32a0b admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:22.367039 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e0f0d-c65a-488d-858a-acbb5d98ad3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:22.367651 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e0f0d-c65a-488d-858a-acbb5d98ad3a admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:22.367927 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e0f0d-c65a-488d-858a-acbb5d98ad3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:22.368186 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e0f0d-c65a-488d-858a-acbb5d98ad3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:22.368814 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e0f0d-c65a-488d-858a-acbb5d98ad3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:22.370404 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e0f0d-c65a-488d-858a-acbb5d98ad3a admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:22.382382 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1300/2605] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 08:11:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 08:11:22.387619 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8fa7f-0ac0-4a89-98f5-e266b819b80c admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:22.397923 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8fa7f-0ac0-4a89-98f5-e266b819b80c admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:22.402739 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829e5e5-5ae4-4096-8643-d440c2fb4955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:22.403384 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829e5e5-5ae4-4096-8643-d440c2fb4955 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:22.403653 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829e5e5-5ae4-4096-8643-d440c2fb4955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:22.403927 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829e5e5-5ae4-4096-8643-d440c2fb4955 admin admin] PATH_INFO: `/v3/projects/3edbb13942c44e69b8fd8fef8d9adba8/users/d77887795cc8422b8c4b504729d70f58/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:22.417035 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:11:22.420234 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d829e5e5-5ae4-4096-8643-d440c2fb4955 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3edbb13942c44e69b8fd8fef8d9adba8, user_id=d77887795cc8422b8c4b504729d70f58, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:22.423221 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d829e5e5-5ae4-4096-8643-d440c2fb4955 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:22.439946 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1306/2606] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 08:11:22 2022] PUT /identity/v3/projects/3edbb13942c44e69b8fd8fef8d9adba8/users/d77887795cc8422b8c4b504729d70f58/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:11:22.444076 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e0f0d-c65a-488d-858a-acbb5d98ad3a admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:22.454360 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e0f0d-c65a-488d-858a-acbb5d98ad3a admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:22.498941 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:11:22.515699 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65049df-cc71-420f-aa19-aaa67363e819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:22.516305 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65049df-cc71-420f-aa19-aaa67363e819 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:22.516573 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65049df-cc71-420f-aa19-aaa67363e819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:22.516864 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65049df-cc71-420f-aa19-aaa67363e819 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:22.517471 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65049df-cc71-420f-aa19-aaa67363e819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:22.519120 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65049df-cc71-420f-aa19-aaa67363e819 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:22.529078 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1301/2607] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 08:11:22 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 08:11:22.533772 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829e5e5-5ae4-4096-8643-d440c2fb4955 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:22.544541 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829e5e5-5ae4-4096-8643-d440c2fb4955 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:22.549562 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25f9ae-a5c8-45e1-8811-a0045cf595f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:22.550213 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25f9ae-a5c8-45e1-8811-a0045cf595f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:22.550506 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25f9ae-a5c8-45e1-8811-a0045cf595f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:22.550805 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25f9ae-a5c8-45e1-8811-a0045cf595f0 admin admin] PATH_INFO: `/v3/projects/3edbb13942c44e69b8fd8fef8d9adba8/users/d77887795cc8422b8c4b504729d70f58/roles/1e35bb725eb8475ea63e2d9538ab67d5` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:22.552849 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed25f9ae-a5c8-45e1-8811-a0045cf595f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3edbb13942c44e69b8fd8fef8d9adba8, user_id=d77887795cc8422b8c4b504729d70f58, role_id=1e35bb725eb8475ea63e2d9538ab67d5)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:22.555525 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed25f9ae-a5c8-45e1-8811-a0045cf595f0 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:22.566144 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1307/2608] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 08:11:22 2022] PUT /identity/v3/projects/3edbb13942c44e69b8fd8fef8d9adba8/users/d77887795cc8422b8c4b504729d70f58/roles/1e35bb725eb8475ea63e2d9538ab67d5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:11:22.581320 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a9594-600a-470e-9cc1-d508f0ec393d None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:22.581483 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a9594-600a-470e-9cc1-d508f0ec393d None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:22.581943 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a9594-600a-470e-9cc1-d508f0ec393d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:22.616435 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-184a9594-600a-470e-9cc1-d508f0ec393d None None] MFA Rules not processed for user `d77887795cc8422b8c4b504729d70f58`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:11:22.617734 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-184a9594-600a-470e-9cc1-d508f0ec393d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:11:22.687793 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1302/2609] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:11:22 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 08:11:22.699478 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25f9ae-a5c8-45e1-8811-a0045cf595f0 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:22.701491 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-ed25f9ae-a5c8-45e1-8811-a0045cf595f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:11:22.761676 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25f9ae-a5c8-45e1-8811-a0045cf595f0 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:22.765968 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73431016-f26f-446e-bdd7-8f10c60371b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:22.766552 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73431016-f26f-446e-bdd7-8f10c60371b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:22.766870 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73431016-f26f-446e-bdd7-8f10c60371b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:22.767138 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73431016-f26f-446e-bdd7-8f10c60371b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:22.829069 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73431016-f26f-446e-bdd7-8f10c60371b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:22.830512 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73431016-f26f-446e-bdd7-8f10c60371b7 service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:22.839746 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1308/2610] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:11:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:11:24.394885 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a9594-600a-470e-9cc1-d508f0ec393d None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:24.396338 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-184a9594-600a-470e-9cc1-d508f0ec393d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:11:24.408789 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a9594-600a-470e-9cc1-d508f0ec393d None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:24.412768 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9780c1-b1e2-4f5f-8a9b-abe6ac3e4dad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:24.413300 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9780c1-b1e2-4f5f-8a9b-abe6ac3e4dad service cinder] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:24.413574 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9780c1-b1e2-4f5f-8a9b-abe6ac3e4dad service cinder] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:24.413839 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9780c1-b1e2-4f5f-8a9b-abe6ac3e4dad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:24.482182 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9780c1-b1e2-4f5f-8a9b-abe6ac3e4dad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:24.484522 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9780c1-b1e2-4f5f-8a9b-abe6ac3e4dad service cinder] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:24.494528 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1303/2611] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:11:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:11:25.107576 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73431016-f26f-446e-bdd7-8f10c60371b7 service cinder] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:25.117202 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73431016-f26f-446e-bdd7-8f10c60371b7 service cinder] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:25.121684 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2030a2-f828-4552-a17b-49d666da6844 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:25.122311 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2030a2-f828-4552-a17b-49d666da6844 service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:25.122583 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2030a2-f828-4552-a17b-49d666da6844 service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:25.122875 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2030a2-f828-4552-a17b-49d666da6844 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:25.132799 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2030a2-f828-4552-a17b-49d666da6844 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:25.134223 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2030a2-f828-4552-a17b-49d666da6844 service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:25.144059 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1309/2612] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:11:25 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:11:29.155830 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9780c1-b1e2-4f5f-8a9b-abe6ac3e4dad service cinder] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:29.225794 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9780c1-b1e2-4f5f-8a9b-abe6ac3e4dad service cinder] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:29.231552 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555db87-96a4-4ea3-b092-1afbc11723a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:29.232178 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555db87-96a4-4ea3-b092-1afbc11723a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:29.232453 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555db87-96a4-4ea3-b092-1afbc11723a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:29.232716 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555db87-96a4-4ea3-b092-1afbc11723a3 admin admin] PATH_INFO: `/v3/users/c461c67cd0b64d5fa14cf6e3bc6203d4` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:29.233876 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d555db87-96a4-4ea3-b092-1afbc11723a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c461c67cd0b64d5fa14cf6e3bc6203d4)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:29.235865 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d555db87-96a4-4ea3-b092-1afbc11723a3 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:29.241290 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=100 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:11:29.292707 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-d555db87-96a4-4ea3-b092-1afbc11723a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c461c67cd0b64d5fa14cf6e3bc6203d4'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:29.293380 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-d555db87-96a4-4ea3-b092-1afbc11723a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c461c67cd0b64d5fa14cf6e3bc6203d4'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:29.300559 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-d555db87-96a4-4ea3-b092-1afbc11723a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c461c67cd0b64d5fa14cf6e3bc6203d4'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:29.314370 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-d555db87-96a4-4ea3-b092-1afbc11723a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c461c67cd0b64d5fa14cf6e3bc6203d4'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:29.324423 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-d555db87-96a4-4ea3-b092-1afbc11723a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c461c67cd0b64d5fa14cf6e3bc6203d4'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:29.332318 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1304/2613] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 08:11:29 2022] DELETE /identity/v3/users/c461c67cd0b64d5fa14cf6e3bc6203d4 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:11:29.336940 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca6fc1-4e3d-40a7-b3f8-751bfeb6f8fa None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:29.337409 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca6fc1-4e3d-40a7-b3f8-751bfeb6f8fa None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:29.337531 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca6fc1-4e3d-40a7-b3f8-751bfeb6f8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:29.357516 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-72ca6fc1-4e3d-40a7-b3f8-751bfeb6f8fa None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:11:29.358763 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-72ca6fc1-4e3d-40a7-b3f8-751bfeb6f8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:11:29.423614 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1310/2614] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:11:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 08:11:29.433652 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555db87-96a4-4ea3-b092-1afbc11723a3 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:29.435199 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-d555db87-96a4-4ea3-b092-1afbc11723a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:11:29.504836 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555db87-96a4-4ea3-b092-1afbc11723a3 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:29.508805 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249394e-6a9a-43d1-a9f2-0f4677f49d81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:29.509458 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249394e-6a9a-43d1-a9f2-0f4677f49d81 service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:29.509733 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249394e-6a9a-43d1-a9f2-0f4677f49d81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:29.510001 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249394e-6a9a-43d1-a9f2-0f4677f49d81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:29.519998 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6249394e-6a9a-43d1-a9f2-0f4677f49d81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:29.521422 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6249394e-6a9a-43d1-a9f2-0f4677f49d81 service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:29.530985 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1305/2615] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:11:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:11:29.822286 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca6fc1-4e3d-40a7-b3f8-751bfeb6f8fa None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:29.823809 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-72ca6fc1-4e3d-40a7-b3f8-751bfeb6f8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:11:29.838984 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca6fc1-4e3d-40a7-b3f8-751bfeb6f8fa None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:29.844945 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44354a5a-d6ac-4354-b8fd-2ca41fba16cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:29.845591 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44354a5a-d6ac-4354-b8fd-2ca41fba16cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:29.845867 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44354a5a-d6ac-4354-b8fd-2ca41fba16cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:29.846138 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44354a5a-d6ac-4354-b8fd-2ca41fba16cf admin admin] PATH_INFO: `/v3/projects/fabd0ccc1f254228aae5e15bf373409f` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:29.848719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44354a5a-d6ac-4354-b8fd-2ca41fba16cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=fabd0ccc1f254228aae5e15bf373409f)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:29.850687 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44354a5a-d6ac-4354-b8fd-2ca41fba16cf admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:29.867910 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:11:29.902763 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-44354a5a-d6ac-4354-b8fd-2ca41fba16cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fabd0ccc1f254228aae5e15bf373409f', 'request_id': 'req-44354a5a-d6ac-4354-b8fd-2ca41fba16cf'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:29.911452 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-44354a5a-d6ac-4354-b8fd-2ca41fba16cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fabd0ccc1f254228aae5e15bf373409f', 'request_id': 'req-44354a5a-d6ac-4354-b8fd-2ca41fba16cf'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:29.919765 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-44354a5a-d6ac-4354-b8fd-2ca41fba16cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fabd0ccc1f254228aae5e15bf373409f', 'request_id': 'req-44354a5a-d6ac-4354-b8fd-2ca41fba16cf'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:29.932169 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-44354a5a-d6ac-4354-b8fd-2ca41fba16cf admin admin] The token cache is being invalidate because project fabd0ccc1f254228aae5e15bf373409f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 08:11:29.932456 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-44354a5a-d6ac-4354-b8fd-2ca41fba16cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:29.933119 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-44354a5a-d6ac-4354-b8fd-2ca41fba16cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:29.934615 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1311/2616] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 08:11:29 2022] DELETE /identity/v3/projects/fabd0ccc1f254228aae5e15bf373409f => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:11:29.938833 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249394e-6a9a-43d1-a9f2-0f4677f49d81 service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:29.948728 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249394e-6a9a-43d1-a9f2-0f4677f49d81 service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:30.010595 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62059ef-16a7-45a0-b3d3-7ecb09e9ab94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:30.011309 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62059ef-16a7-45a0-b3d3-7ecb09e9ab94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:30.011580 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62059ef-16a7-45a0-b3d3-7ecb09e9ab94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:30.011847 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62059ef-16a7-45a0-b3d3-7ecb09e9ab94 admin admin] PATH_INFO: `/v3/users/039badaf49a74b4c970196f7b191c1b8` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:30.013065 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62059ef-16a7-45a0-b3d3-7ecb09e9ab94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=039badaf49a74b4c970196f7b191c1b8)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:30.015079 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62059ef-16a7-45a0-b3d3-7ecb09e9ab94 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:30.045893 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:11:30.064777 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c62059ef-16a7-45a0-b3d3-7ecb09e9ab94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '039badaf49a74b4c970196f7b191c1b8'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:30.065501 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c62059ef-16a7-45a0-b3d3-7ecb09e9ab94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '039badaf49a74b4c970196f7b191c1b8'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:30.072350 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c62059ef-16a7-45a0-b3d3-7ecb09e9ab94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '039badaf49a74b4c970196f7b191c1b8'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:30.087922 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c62059ef-16a7-45a0-b3d3-7ecb09e9ab94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '039badaf49a74b4c970196f7b191c1b8'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:30.099752 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-c62059ef-16a7-45a0-b3d3-7ecb09e9ab94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '039badaf49a74b4c970196f7b191c1b8'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:30.106639 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1306/2617] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 08:11:29 2022] DELETE /identity/v3/users/039badaf49a74b4c970196f7b191c1b8 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:11:30.413273 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44354a5a-d6ac-4354-b8fd-2ca41fba16cf admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:30.414828 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-44354a5a-d6ac-4354-b8fd-2ca41fba16cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:11:30.429673 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44354a5a-d6ac-4354-b8fd-2ca41fba16cf admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:30.491345 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca011c3-748f-4092-9a6f-0d65be0c3333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:30.492020 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca011c3-748f-4092-9a6f-0d65be0c3333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:30.492334 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca011c3-748f-4092-9a6f-0d65be0c3333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:30.492591 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca011c3-748f-4092-9a6f-0d65be0c3333 admin admin] PATH_INFO: `/v3/projects/e3110822fe994d0594692b628df8ec87` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:30.493785 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca011c3-748f-4092-9a6f-0d65be0c3333 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3110822fe994d0594692b628df8ec87)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:30.495790 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca011c3-748f-4092-9a6f-0d65be0c3333 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:30.545749 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-9ca011c3-748f-4092-9a6f-0d65be0c3333 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3110822fe994d0594692b628df8ec87', 'request_id': 'req-9ca011c3-748f-4092-9a6f-0d65be0c3333'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:30.554394 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-9ca011c3-748f-4092-9a6f-0d65be0c3333 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3110822fe994d0594692b628df8ec87', 'request_id': 'req-9ca011c3-748f-4092-9a6f-0d65be0c3333'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:30.560267 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-9ca011c3-748f-4092-9a6f-0d65be0c3333 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3110822fe994d0594692b628df8ec87', 'request_id': 'req-9ca011c3-748f-4092-9a6f-0d65be0c3333'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:30.571601 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-9ca011c3-748f-4092-9a6f-0d65be0c3333 admin admin] The token cache is being invalidate because project e3110822fe994d0594692b628df8ec87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 08:11:30.571885 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-9ca011c3-748f-4092-9a6f-0d65be0c3333 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:30.572507 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-9ca011c3-748f-4092-9a6f-0d65be0c3333 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:30.574002 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1312/2618] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 08:11:30 2022] DELETE /identity/v3/projects/e3110822fe994d0594692b628df8ec87 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:11:30.578174 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62059ef-16a7-45a0-b3d3-7ecb09e9ab94 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:30.579724 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-c62059ef-16a7-45a0-b3d3-7ecb09e9ab94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:11:30.592924 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62059ef-16a7-45a0-b3d3-7ecb09e9ab94 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:30.653832 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37c4b7d-8a72-4d96-847d-42be64ba367d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:30.654452 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37c4b7d-8a72-4d96-847d-42be64ba367d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:30.654753 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37c4b7d-8a72-4d96-847d-42be64ba367d admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:30.655079 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37c4b7d-8a72-4d96-847d-42be64ba367d admin admin] PATH_INFO: `/v3/users/d77887795cc8422b8c4b504729d70f58` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:30.656140 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37c4b7d-8a72-4d96-847d-42be64ba367d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d77887795cc8422b8c4b504729d70f58)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:30.658053 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37c4b7d-8a72-4d96-847d-42be64ba367d admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:30.712595 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-e37c4b7d-8a72-4d96-847d-42be64ba367d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd77887795cc8422b8c4b504729d70f58'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:30.713378 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-e37c4b7d-8a72-4d96-847d-42be64ba367d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd77887795cc8422b8c4b504729d70f58'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:30.721975 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-e37c4b7d-8a72-4d96-847d-42be64ba367d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd77887795cc8422b8c4b504729d70f58'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:30.736476 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-e37c4b7d-8a72-4d96-847d-42be64ba367d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd77887795cc8422b8c4b504729d70f58'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:30.749154 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-e37c4b7d-8a72-4d96-847d-42be64ba367d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd77887795cc8422b8c4b504729d70f58'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:30.755891 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1307/2619] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 08:11:30 2022] DELETE /identity/v3/users/d77887795cc8422b8c4b504729d70f58 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:11:31.057922 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca011c3-748f-4092-9a6f-0d65be0c3333 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:31.060050 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-9ca011c3-748f-4092-9a6f-0d65be0c3333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:11:31.077950 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca011c3-748f-4092-9a6f-0d65be0c3333 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:31.111006 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:11:31.143373 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3be9fc-5304-4bcd-a7a2-a0235ebcc870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:31.144091 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3be9fc-5304-4bcd-a7a2-a0235ebcc870 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:31.145053 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3be9fc-5304-4bcd-a7a2-a0235ebcc870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:31.145053 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3be9fc-5304-4bcd-a7a2-a0235ebcc870 admin admin] PATH_INFO: `/v3/projects/3edbb13942c44e69b8fd8fef8d9adba8` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:31.145836 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3be9fc-5304-4bcd-a7a2-a0235ebcc870 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3edbb13942c44e69b8fd8fef8d9adba8)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:31.147719 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3be9fc-5304-4bcd-a7a2-a0235ebcc870 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:31.203141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5f3be9fc-5304-4bcd-a7a2-a0235ebcc870 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3edbb13942c44e69b8fd8fef8d9adba8', 'request_id': 'req-5f3be9fc-5304-4bcd-a7a2-a0235ebcc870'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:31.214571 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5f3be9fc-5304-4bcd-a7a2-a0235ebcc870 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3edbb13942c44e69b8fd8fef8d9adba8', 'request_id': 'req-5f3be9fc-5304-4bcd-a7a2-a0235ebcc870'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:31.222904 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5f3be9fc-5304-4bcd-a7a2-a0235ebcc870 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3edbb13942c44e69b8fd8fef8d9adba8', 'request_id': 'req-5f3be9fc-5304-4bcd-a7a2-a0235ebcc870'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:31.237309 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5f3be9fc-5304-4bcd-a7a2-a0235ebcc870 admin admin] The token cache is being invalidate because project 3edbb13942c44e69b8fd8fef8d9adba8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 08:11:31.237309 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5f3be9fc-5304-4bcd-a7a2-a0235ebcc870 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:31.238076 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-5f3be9fc-5304-4bcd-a7a2-a0235ebcc870 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:31.240953 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1313/2620] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 08:11:31 2022] DELETE /identity/v3/projects/3edbb13942c44e69b8fd8fef8d9adba8 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:11:31.247618 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d00c88-e7ae-403d-bcdd-cd6eae5ecd5a None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:31.247883 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d00c88-e7ae-403d-bcdd-cd6eae5ecd5a None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:31.248192 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d00c88-e7ae-403d-bcdd-cd6eae5ecd5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:31.274401 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-e4d00c88-e7ae-403d-bcdd-cd6eae5ecd5a None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:11:31.275689 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-e4d00c88-e7ae-403d-bcdd-cd6eae5ecd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:11:31.301508 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:11:31.353460 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1308/2621] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:11:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 08:11:31.366029 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e364b0-f409-43fb-b078-d3e008320a8c None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:31.366312 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e364b0-f409-43fb-b078-d3e008320a8c None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:31.366555 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e364b0-f409-43fb-b078-d3e008320a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:31.393777 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-81e364b0-f409-43fb-b078-d3e008320a8c None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:11:31.395061 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-81e364b0-f409-43fb-b078-d3e008320a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:11:31.406305 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1314/2622] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:11:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 08:11:31.422082 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d00c88-e7ae-403d-bcdd-cd6eae5ecd5a None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:31.433627 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d00c88-e7ae-403d-bcdd-cd6eae5ecd5a None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:31.439890 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1acd2f-ac36-4e51-851c-3c0eb3e55268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:31.440517 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1acd2f-ac36-4e51-851c-3c0eb3e55268 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:31.440791 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1acd2f-ac36-4e51-851c-3c0eb3e55268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:31.441058 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1acd2f-ac36-4e51-851c-3c0eb3e55268 admin admin] PATH_INFO: `/v3/domains` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:31.441705 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1acd2f-ac36-4e51-851c-3c0eb3e55268 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 08:11:31.441986 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1acd2f-ac36-4e51-851c-3c0eb3e55268 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:31.443737 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1acd2f-ac36-4e51-851c-3c0eb3e55268 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:31.458136 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1309/2623] 10.222.0.30 () {58 vars in 1153 bytes} [Wed Nov 2 08:11:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 08:11:31.467708 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e364b0-f409-43fb-b078-d3e008320a8c None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:31.477828 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e364b0-f409-43fb-b078-d3e008320a8c None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:31.484161 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa8c54-81ca-4630-b848-6648c8f493f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:31.484782 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa8c54-81ca-4630-b848-6648c8f493f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:31.486366 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa8c54-81ca-4630-b848-6648c8f493f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:31.486366 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa8c54-81ca-4630-b848-6648c8f493f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:31.486366 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa8c54-81ca-4630-b848-6648c8f493f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:31.488994 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa8c54-81ca-4630-b848-6648c8f493f1 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:31.529236 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1315/2624] 10.222.0.30 () {60 vars in 1153 bytes} [Wed Nov 2 08:11:31 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 08:11:31.533825 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1acd2f-ac36-4e51-851c-3c0eb3e55268 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:31.544750 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1acd2f-ac36-4e51-851c-3c0eb3e55268 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:31.617220 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e09bb24-22ac-40e6-8d45-ef062f6e60c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:31.617868 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e09bb24-22ac-40e6-8d45-ef062f6e60c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:31.618122 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e09bb24-22ac-40e6-8d45-ef062f6e60c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:31.618366 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e09bb24-22ac-40e6-8d45-ef062f6e60c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:31.619062 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e09bb24-22ac-40e6-8d45-ef062f6e60c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:31.621877 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e09bb24-22ac-40e6-8d45-ef062f6e60c8 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:31.650872 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1310/2625] 10.222.0.30 () {60 vars in 1144 bytes} [Wed Nov 2 08:11:31 2022] POST /identity/v3/users => generated 287 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 08:11:31.664087 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa8c54-81ca-4630-b848-6648c8f493f1 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:31.679655 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa8c54-81ca-4630-b848-6648c8f493f1 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:31.690213 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b18bc5b-f522-473d-b34c-541d512a7f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:31.690911 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18bc5b-f522-473d-b34c-541d512a7f35 admin admin] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:31.691179 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18bc5b-f522-473d-b34c-541d512a7f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:31.691448 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18bc5b-f522-473d-b34c-541d512a7f35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:31.692105 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b18bc5b-f522-473d-b34c-541d512a7f35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:31.693732 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b18bc5b-f522-473d-b34c-541d512a7f35 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:31.706200 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1316/2626] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 08:11:31 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 08:11:31.711126 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e09bb24-22ac-40e6-8d45-ef062f6e60c8 admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:31.724780 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e09bb24-22ac-40e6-8d45-ef062f6e60c8 admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:31.730058 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a0fb90-e2a9-4b50-b0d6-25a607012223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:31.730694 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0fb90-e2a9-4b50-b0d6-25a607012223 admin admin] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:31.730984 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0fb90-e2a9-4b50-b0d6-25a607012223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:31.731536 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0fb90-e2a9-4b50-b0d6-25a607012223 admin admin] PATH_INFO: `/v3/roles` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:31.732193 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a0fb90-e2a9-4b50-b0d6-25a607012223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:31.733806 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a0fb90-e2a9-4b50-b0d6-25a607012223 admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:31.747390 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1311/2627] 10.222.0.30 () {58 vars in 1122 bytes} [Wed Nov 2 08:11:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 08:11:31.752095 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b18bc5b-f522-473d-b34c-541d512a7f35 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:31.769136 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b18bc5b-f522-473d-b34c-541d512a7f35 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:31.775792 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75481c68-3afc-453c-9417-f7e9f11341c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:31.776676 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75481c68-3afc-453c-9417-f7e9f11341c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:31.777141 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75481c68-3afc-453c-9417-f7e9f11341c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:31.777588 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75481c68-3afc-453c-9417-f7e9f11341c5 admin admin] PATH_INFO: `/v3/projects/350e9d6183564773bf03526c6e90d7b2/users/40bba4758cfc429ab4690fbe8b19038f/roles/15930dae1c0b464f98680edb03b7f02c` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:31.796538 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75481c68-3afc-453c-9417-f7e9f11341c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=350e9d6183564773bf03526c6e90d7b2, user_id=40bba4758cfc429ab4690fbe8b19038f, role_id=15930dae1c0b464f98680edb03b7f02c)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:31.799291 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75481c68-3afc-453c-9417-f7e9f11341c5 admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:31.814520 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1317/2628] 10.222.0.30 () {60 vars in 1483 bytes} [Wed Nov 2 08:11:31 2022] PUT /identity/v3/projects/350e9d6183564773bf03526c6e90d7b2/users/40bba4758cfc429ab4690fbe8b19038f/roles/15930dae1c0b464f98680edb03b7f02c => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:11:31.827702 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6a036-3b2f-4307-90cb-aa42a6f49afe None None] REQUEST_METHOD: `POST` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:31.827955 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6a036-3b2f-4307-90cb-aa42a6f49afe None None] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:31.828254 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6a036-3b2f-4307-90cb-aa42a6f49afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:31.869120 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.auth.core [None req-33f6a036-3b2f-4307-90cb-aa42a6f49afe None None] MFA Rules not processed for user `40bba4758cfc429ab4690fbe8b19038f`. Rule list: `[]` (Enabled: `True`). {{(pid=81548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:11:31.871064 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-33f6a036-3b2f-4307-90cb-aa42a6f49afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:11:31.946538 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1312/2629] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:11:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 08:11:31.958539 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75481c68-3afc-453c-9417-f7e9f11341c5 admin admin] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:31.960531 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-75481c68-3afc-453c-9417-f7e9f11341c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:11:32.026558 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75481c68-3afc-453c-9417-f7e9f11341c5 admin admin] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:32.032050 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e880e67e-7b28-4742-a8ec-76365868dc50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '315861510b1041dca19d1e8753d9b21d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:32.032682 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e880e67e-7b28-4742-a8ec-76365868dc50 service cinder] REQUEST_METHOD: `GET` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:32.032960 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e880e67e-7b28-4742-a8ec-76365868dc50 service cinder] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:32.033227 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e880e67e-7b28-4742-a8ec-76365868dc50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:32.043429 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e880e67e-7b28-4742-a8ec-76365868dc50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:32.044942 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e880e67e-7b28-4742-a8ec-76365868dc50 service cinder] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:32.054840 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1318/2630] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:11:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:11:41.943376 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=88,keystone:INSERT=3 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:11:42.050468 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:11:42.324802 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6a036-3b2f-4307-90cb-aa42a6f49afe None None] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:42.392255 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6a036-3b2f-4307-90cb-aa42a6f49afe None None] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:42.397497 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5480c4-93ca-443c-874e-1cbf37bf63ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:42.398114 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5480c4-93ca-443c-874e-1cbf37bf63ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:42.398384 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5480c4-93ca-443c-874e-1cbf37bf63ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:42.398651 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5480c4-93ca-443c-874e-1cbf37bf63ab admin admin] PATH_INFO: `/v3/users/40bba4758cfc429ab4690fbe8b19038f` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:42.399797 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5480c4-93ca-443c-874e-1cbf37bf63ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=40bba4758cfc429ab4690fbe8b19038f)` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:42.401728 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5480c4-93ca-443c-874e-1cbf37bf63ab admin admin] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:42.461127 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-4b5480c4-93ca-443c-874e-1cbf37bf63ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40bba4758cfc429ab4690fbe8b19038f'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:42.461797 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-4b5480c4-93ca-443c-874e-1cbf37bf63ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40bba4758cfc429ab4690fbe8b19038f'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:42.469166 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-4b5480c4-93ca-443c-874e-1cbf37bf63ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40bba4758cfc429ab4690fbe8b19038f'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:42.482125 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-4b5480c4-93ca-443c-874e-1cbf37bf63ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40bba4758cfc429ab4690fbe8b19038f'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:42.492399 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.notifications [None req-4b5480c4-93ca-443c-874e-1cbf37bf63ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40bba4758cfc429ab4690fbe8b19038f'} {{(pid=81548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:42.498574 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1313/2631] 10.222.0.30 () {58 vars in 1224 bytes} [Wed Nov 2 08:11:42 2022] DELETE /identity/v3/users/40bba4758cfc429ab4690fbe8b19038f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:11:42.503356 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0274623-8de0-4bd0-a680-d88b1a7e1131 None None] REQUEST_METHOD: `POST` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:42.503640 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0274623-8de0-4bd0-a680-d88b1a7e1131 None None] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:42.503907 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0274623-8de0-4bd0-a680-d88b1a7e1131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:42.525285 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.auth.core [None req-c0274623-8de0-4bd0-a680-d88b1a7e1131 None None] MFA Rules not processed for user `8715d21c18b14387953f25f094b18c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 08:11:42.526506 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-c0274623-8de0-4bd0-a680-d88b1a7e1131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:11:42.589191 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1319/2632] 10.222.0.30 () {58 vars in 958 bytes} [Wed Nov 2 08:11:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 08:11:42.601714 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5480c4-93ca-443c-874e-1cbf37bf63ab admin admin] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:42.603372 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-4b5480c4-93ca-443c-874e-1cbf37bf63ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:11:42.664702 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5480c4-93ca-443c-874e-1cbf37bf63ab admin admin] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:42.668783 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50042e5c-9111-4fd6-b688-bd00b43a75ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d51ec74e1e4c25bcc4365eb858bac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:42.669393 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50042e5c-9111-4fd6-b688-bd00b43a75ba service neutron] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:42.669664 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50042e5c-9111-4fd6-b688-bd00b43a75ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:42.669929 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50042e5c-9111-4fd6-b688-bd00b43a75ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:42.679510 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50042e5c-9111-4fd6-b688-bd00b43a75ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:42.681008 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50042e5c-9111-4fd6-b688-bd00b43a75ba service neutron] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:42.690332 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1314/2633] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:11:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:11:42.961800 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0274623-8de0-4bd0-a680-d88b1a7e1131 None None] Authenticating user token {{(pid=81549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:11:42.963307 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.fernet_utils [None req-c0274623-8de0-4bd0-a680-d88b1a7e1131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:11:42.976948 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0274623-8de0-4bd0-a680-d88b1a7e1131 None None] Validating token access rules against request {{(pid=81549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:11:42.982777 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9931a-a3dc-4789-9905-c1cb2d0c903e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8715d21c18b14387953f25f094b18c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b3f647c33aef4316a529d7730281cb6b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:11:42.983433 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9931a-a3dc-4789-9905-c1cb2d0c903e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:11:42.984373 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9931a-a3dc-4789-9905-c1cb2d0c903e admin admin] SCRIPT_NAME: `/identity` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:11:42.984677 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9931a-a3dc-4789-9905-c1cb2d0c903e admin admin] PATH_INFO: `/v3/projects/350e9d6183564773bf03526c6e90d7b2` {{(pid=81549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:11:42.986052 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a9931a-a3dc-4789-9905-c1cb2d0c903e admin admin] RBAC: Authorizing `identity:delete_project(project_id=350e9d6183564773bf03526c6e90d7b2)` {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:11:42.988016 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a9931a-a3dc-4789-9905-c1cb2d0c903e admin admin] RBAC: Authorization granted {{(pid=81549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:11:43.041257 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-94a9931a-a3dc-4789-9905-c1cb2d0c903e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '350e9d6183564773bf03526c6e90d7b2', 'request_id': 'req-94a9931a-a3dc-4789-9905-c1cb2d0c903e'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:43.051475 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-94a9931a-a3dc-4789-9905-c1cb2d0c903e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '350e9d6183564773bf03526c6e90d7b2', 'request_id': 'req-94a9931a-a3dc-4789-9905-c1cb2d0c903e'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:43.057063 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-94a9931a-a3dc-4789-9905-c1cb2d0c903e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '350e9d6183564773bf03526c6e90d7b2', 'request_id': 'req-94a9931a-a3dc-4789-9905-c1cb2d0c903e'} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:43.068300 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-94a9931a-a3dc-4789-9905-c1cb2d0c903e admin admin] The token cache is being invalidate because project 350e9d6183564773bf03526c6e90d7b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 08:11:43.068587 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-94a9931a-a3dc-4789-9905-c1cb2d0c903e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:43.069279 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG keystone.notifications [None req-94a9931a-a3dc-4789-9905-c1cb2d0c903e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 08:11:43.071017 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: [pid: 81549|app: 0|req: 1320/2634] 10.222.0.30 () {58 vars in 1233 bytes} [Wed Nov 2 08:11:42 2022] DELETE /identity/v3/projects/350e9d6183564773bf03526c6e90d7b2 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 08:11:52.686413 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:11:53.065117 dv-cinder-804757-31-iscsi devstack@keystone.service[81549]: DEBUG dbcounter [-] [81549] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 08:12:36.425390 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50042e5c-9111-4fd6-b688-bd00b43a75ba service neutron] Authenticating user token {{(pid=81548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 08:12:36.427257 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-50042e5c-9111-4fd6-b688-bd00b43a75ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:12:36.514199 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50042e5c-9111-4fd6-b688-bd00b43a75ba service neutron] Validating token access rules against request {{(pid=81548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 08:12:36.576431 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d415a9c-6bf4-4e78-855f-b06a4bf03b6d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '932d3076fb4b426d8c83d02eb60d8063', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25e2bc8417754f909b416b93d5b7afb7', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 08:12:36.577344 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d415a9c-6bf4-4e78-855f-b06a4bf03b6d service placement] REQUEST_METHOD: `GET` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 08:12:36.577755 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d415a9c-6bf4-4e78-855f-b06a4bf03b6d service placement] SCRIPT_NAME: `/identity` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 08:12:36.578148 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d415a9c-6bf4-4e78-855f-b06a4bf03b6d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 08:12:36.581572 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.fernet_utils [None req-3d415a9c-6bf4-4e78-855f-b06a4bf03b6d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 08:12:36.596486 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d415a9c-6bf4-4e78-855f-b06a4bf03b6d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 08:12:36.598090 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d415a9c-6bf4-4e78-855f-b06a4bf03b6d service placement] RBAC: Authorization granted {{(pid=81548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 08:12:36.609371 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: [pid: 81548|app: 0|req: 1315/2635] 10.222.0.30 () {60 vars in 1365 bytes} [Wed Nov 2 08:12:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 08:12:46.604349 dv-cinder-804757-31-iscsi devstack@keystone.service[81548]: DEBUG dbcounter [-] [81548] Writing DB stats keystone:SELECT=63 {{(pid=81548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}